Return to instructor resources


Data collection instruments for assessing
development of intercultural competence


How do we assess intercultural communicative competence among second language learners? The development of intercultural communicative competence is a challenging research topic that interests me. I am looking at models, frameworks and surveys that researchers have developed and used in specific contexts. My goal for  this webpage is to show what I have found in research, and develop a valid rubric, questionnaire and survey that will measure student intercultural competence that is developed through online inquiry and critical discourse analysis of native forums through collaborative blogging with peers. These tools are specially tuned for my research project.


 Questions for Teacher-Student post-treatment interview

The manner in that this post-treatment interview is administered is the following. First the participant responds to the statements. After responding, the researcher asks the participant to explain through a casual interview. The intention is to understand why some students would score low on the blog rubric, showing a minimal online activity. A more sophisticated interview would probably need to be developed.








Competencies
1
Highly Disagree
 2
Disagree

 3
Indifferent

4
Agree
5
Highly Agree

Explain
Attitude Shift
1) My attitudes and ideas towards the foreign culture have become more positive.
2) I am more aware of the differences and relationships between my culture and the foreign culture.






Skills of Observation and Discovery
1) I have successfully observed and analyzed how people of another language and culture perceive and experience the world.
2) I become aware of the beliefs, values and meanings that are shared among my culture and the foreign culture.






Cultural Knowledge
1) I became aware of beliefs, values and meanings of the native culture.
























Rubric for assessing development of intercultural competence of
pre-sojourn student-to-student blog postings


This is the rubric that I used for valuing each blog posting. I specially adapted this framework for the peer-to-peer blogging context from the categories and descriptions of the Intercultural Competence Assessment (INCA) project (2004) and 
the intercultural constructs from Byram (1997).

Sources:     For information on the INCA project and rubric go to  www.incaproject.org/framework.htm  Click on "assessor                    version" to see the rubric.
                   Byram, M. (1997). Teaching and assessing intercultural communicative competence. Clevedon: Multilingual                        Matters. 


 

Basic (Value = 1)

Intermediate (Value = 2)

Full (Value = 3)


Blog responses are piecemeal and improvised rather than principled. These will be based on fragmentary information.

Simple principles are used to analyze the texts rather than improvised responses. There is evidence of a basic strategy and some coherent knowledge of the texts.

A combination of strategic and principled approaches to discourse analysis are used. The learner takes the role of a mediator among peers in seeks to bring about the most accurate perception from peers.

Knowledge of their own culture and that of others is both coherent and sophisticated.

i) Tolerance of ambiguity

1T

Has no means of approaching ambiguous texts. Expresses intolerance of ambiguous texts found in native forums.

2T

Has begun to acquire a repertoire of approaches to cope with ambiguities. Begins to accept ambiguity as a challenge.

3T

Is constantly aware of the possibility of ambiguity. When it occurs, he/she tolerates and manages it.

ii) Attitude shift

1A

Shares ethnocentric views with classmates and perceives these as valid interpretations. Cultural similarities and differences were not expressed.

2A

Recognizes and begins to question ethnocentric views that are expressed on the forum.

3A

Produces new perceptions that are based on their analysis of  and valued analysis on the student forum. Expresses and understanding of cultural similarities and differences, relying on native text analysis.

iii) Knowledge discovery

1K

Blog entries draw on random general knowledge about the target culture and do not make real reference to native postings nor do they show any reflection on the native forums.

2K

Modifies and builds on information found in forums, in the light of actual experience. Blogs are motivated by curiosity to develop knowledge of his own culture as perceived by others.

3K

Expresses a deep knowledge of the target culture. Develops knowledge through systematic research-like activities and direct questioning and can, where this is sought, offer advice and support to peers in the online discussion.

iv) Empathy

1E

Tends to see the cultural foreigner’s differences as curious, and remains confused about the seemingly strange behaviors and their antecedents. Nonetheless tries to ‘make allowances’.

2E

Has the beginnings of a mental checklist of how others may perceive, feel and respond differently to, a range of routine circumstances. Tends increasingly to see things intuitively from the native point of view.

3E

Accepts the native postings on forms as coherent individuals. Enlists role-taking and de-centering skills and awareness of different perspectives in optimizing

 

INCA Framework (assessor version)

The Hett Global-Mindedness (GMS) Scale (1993)


The following survey will be administered before and after treatment to identify a positive effect of treatment in the research project. This survey has been used in other intercultural competence studies of various professional contexts. It is very general in nature, my use of it is to measure any holistic gain of global-mindedness in the categories below.

These items represent a five dimensions of intercultural competence in relation to global-mindedness, these are: 1) responsibility, 2) cultural pluralism, 3) efficacy, 4) globalcenterism, and 5) interconnectedness.  Since students in this project are not directly interacting with natives, but rather analyzing native forum postings and co-constructing the cultural meaning with classmates, it is assumed that the category of cultural pluralism (Items 1, 3, 8, 13, 14, 19, 24, 27) and globalcenterism (Items 5, 10, 16, 21, 29) will show a shift to a positive distribution in the post-survey. The other categories should not change significantly.



STUDENT ATTITUDE SURVEY

The data collected will be confidential. Your score has no effect on your grade. In the following you will find a series of statements. Please read each statement and decide whether or not you agree or disagree. Circle the response that most accurately reflects your opinion. There are no “correct” answers.


                                                               Strongly Disagree = 1, Disagree = 2, Unsure = 3, Agree = 4, Strongly Agree =5                                                                                     
1. I generally find it stimulating to                                         1                     2                  3                4                             5
spend an evening talking with people from
another culture.

2. I feel an obligation to speak out when                               1                     2                  3                4                             5
I see our government doing something I
consider wrong.

3. The United States is enriched by the                                 1                     2                  3                4                             5
fact that it is comprised of many people from
different cultures and countries.

4. Really, there is nothing I can do about the                        1                     2                  3                4                             5
problems of the world

5. The needs of the United States must                                 1                     2                  3                4                             5
continue to be our highest priority in
negotiating with other countries.

6. I often think about the kind of world we                          1                     2                  3                4                             5
are creating for future generations.

7. When I hear that thousands of people                              1                     2                  3                4                             5
are starving in an African country, I feel
very frustrated.

8. Americans can learn something of value                         1                     2                  3                4                             5
from all different cultures.

9. Generally, an individual’s actions are too                        1                     2                  3                4                             5
small to have a significant effect on the
ecosystem.

10. Americans should be permitted to pursue                     1                     2                  3                4                             5
the standard of living they can afford if it
only has a slight negative impact on the
environment.

11. I think of myself, not only as a citizen of my                1                     2                  3                4                             5
country, but also as a citizen of the world.

12. When I see the conditions some people                         1                     2                  3                4                             5
in the world live under, I feel a responsibility
to do something about it.

13. I enjoy trying to understand people’s                             1                     2                  3                4                             5
behavior in the context of their culture.

14. My opinions about national policies are                        1                     2                  3                4                             5
based on how those policies might affect the
rest of the world as well as the United States.

15. It is very important to me to choose a career                 1                     2                  3                4                             5
in which I can have a positive effect on the
quality of life for future generations.

16. American values are probably the best.                         1                     2                  3                4                             5

17. In the long run, America will probably                         1                     2                  3                4                             5
benefit from the fact that world is becoming
more interconnected.

18. The fact that a flood can kill 50,000                              1                     2                  3                4                             5
people in Bangladesh is very depressing to me.

19. It is important that American universities                     1                     2                  3                4                             5
and colleges provide programs designed to
promote understanding among students of
different ethnic and cultural backgrounds.

20. I think my behavior can impact people in                     1                     2                  3                4                             5
other countries.

21. The present distribution of the world’s                         1                     2                  3                4                             5
wealth and resources should be maintained
because it promotes survival of the fittest.

22. I feel a strong kinship with the worldwide                   1                     2                  3                4                             5
human family.

23. I feel very concerned about the lives of                        1                     2                  3                4                             5
people who live in politically repressive regimes.

24. It is important that we educate people to                      1                     2                  3                4                             5
understand the impact that current policies might
have on future generations.

25. It is not really important to me to consider                   1                     2                  3                4                             5
myself as a member of the global community.

26. I sometimes try to imagine how a person                      1                     2                  3                4                             5
who is always hungry must feel.

27. I have very little in common with people in                  1                     2                  3                4                             5
underdeveloped nations.

28. I am able to affect what happens on a global                 1                     2                  3                4                             5
level by what I do in my own community.

29. I sometimes feel irritated with people from                  1                     2                  3                4                             5
other countries because they don’t understand
how we do things here.

30. Americans have a moral obligation to share                 1                     2                  3                4                             5
their wealth with the less fortunate peoples of
the world.

SCORING PROCEDURE
 
      SCORING KEY:   Reverse score items: 4, 5, 9, 10, 16, 21, 25, 27, 29

      SCORING:          * Range of scores 30 – 150
                                   * Sum all responses
                                   * Higher scores indicate a higher level
                                     of global-mindedness


      ITEMS REFLECTING THEORETICAL DIMENSIONS

      RESPONSIBILITY:                    2, 7, 12, 18, 23, 25, 30
      CULTURAL PLURALISM:       1, 3, 8, 13, 14, 19, 24, 27
      EFFICACY:                                 4, 9, 15, 20, 28
      GLOBALCENTRISM:               5, 10, 16, 21, 29
      INTERCONNECTEDNESS:      6, 11, 17, 22, 25