Assessment for learning: Social science teachers` ideas on

Assessment for learning: Social science
teachers’ ideas on assessment of causal
understanding
J. Alonso-Tapia1, F. Asensio2, I. López3,
N. Carriedo4, F. Rycheck5
1
School of Psychology, Autónoma University of Madrid, Spain
María Zambrano Secondary School, Leganés (Madrid), Spain
3
Joaquín Araujo Secondary School, Fuenlabrada (Madrid), Spain
4
Dept. of Developmental & Educational Psychology,
National University for Distance Education, Madrid, Spain
5
Director General, Facultad Católica de Rondonia, Brazil
2
Spain / Brazil
Jesús Alonso Tapia. Universidad Autónoma de Madrid, Campus de Canto Blanco. 28049 Madrid, Spain. E-mail:
[email protected]
© Education & Psychology I+D+i and Editorial EOS (Spain)
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 593 -
Jesús Alonso-Tapia et al.
Abstract
The main objective of this work, carried out as part of a series aimed at identifying teachers’
ideas on assessment, was to identify social science teachers’ ideas on how causal understanding should be assessed when assessing for learning. After establishing the psychological model from which teachers’ ideas should be evaluated, 68 secondary school teachers were asked
to design assessment situations adequate for the mentioned purpose. Results showed convergence of data coming from Brazil and Spain and enlightened specific characteristics of teachers’ ideas on causal understanding assessment. So they are useful for designing teachers’
training programs on students’ assessment.
Keywords: Causal understanding assessment, Assessment in social sciences, Knowledge assessment, Teachers’ thinking.
Received: 06-07-07
- 594 -
Initial acceptance: 08-10-07
Final acceptance: 11-23-07
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
Introduction
There is growing interest among psychologists and educators in knowing what
personal and environmental factors affect learning motivation, understanding and conceptual
change in order to favour these processes (Alonso-Tapia & Pardo, 2006; Limón & Mason,
2002). According to recent developments (Black & William, 1998; Birembaum, 2003;
Birembaum et al. 2006, Segers, Dochy & Cascallar, 2003) assessment is one of the
environmental factors that most influences learning activities and the promotion of
understanding and knowledge and abilities acquisition. So, in order to improve students’
learning it seems necessary to change not only instruction but also teachers’ perspective on
assessment (Dochy, 2005). Instead of “testing” the amount of knowledge that students have,
assessment should provide information –to the teacher or the student him/herself- that could
help students to overcome their difficulties and to self-regulate their understanding and
learning processes.
However, there is evidence that the preferred ways of assessment of many teachers
have many shortcomings (Black & William, 1998; Segers, Dochy & Cascallar, 2003).
Assessment tasks are often not valid criteria of understanding and cognitive ability, as they
rely mainly on root learning (Crooks, 1988; Villa & Alonso-Tapia, 1996). In fact many tasks
do not allow teachers to identify the source of students’ difficulties, and so feedback lacks the
necessary quality to be of help for overcoming learning problems.
In a recent paper Birembaum et al. (2006) point to the shortcomings of current
assessment practices and suggest alternatives directions to overcome them. However, this
review does not deal with the nature that specific tasks related to different learning objectives
should have in order to promote understanding and conceptual change. This was a problem
examined for us in a previous work (Alonso-Tapia (2002). According to this work, when
trying to promote understanding and conceptual change-assessment contexts should had
several characteristics from which three of them are relevant for the purpose of this paper: a)
Most suitable tasks are those demanding the application and use of knowledge for solving
problems implying some degree of novelty (analogous and transfer tasks); b) tasks should be
designed to allow teachers to identify specific factors in students that hinder understanding; and c)
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 595 -
Jesús Alonso-Tapia et al.
the assessment process should cover the different nodes and links of the conceptual or procedural
network that the students are supposed to have construct.
In the context of these ideas, the following questions arise: 1) what are the characteristics
of assessment tasks actually used by teachers? 2) What are teachers’ ideas about how knowledge
and abilities should best be assessed to favour understanding?
In relation to the first question and within the limits of Social Sciences –the area of our
research-, previous studies had shown that assessment tasks used by secondary and highschool social science teachers demand mainly rote-memory (Villa & Alonso-Tapia, 1996;
Black and William, 1998; Crooks, 1988), though the possible tasks are multiple (AlonsoTapia, 1997). However, according to explicit objectives of the Spanish and Brazilian
curriculum, the study of social sciences should make students able to understand information
contained in different kinds of document and to build causal models useful for assessing
critically present social problems. Nevertheless, neither the mentioned abilities nor the kind of
understanding implied by the causal models build by the students are usually assessed.
The assessment problem just mentioned could be due to the fact that teachers had
inadequate or erroneous ideas about what constitutes a valid indicator of the abilities being
assessed, of causal understanding or of specific factors in students that might hinder
comprehension. If this was the case, teacher training should try to change such ideas.
For example, according to psychology, the ability to understand and extract
information from different kinds of document is shown when assessment tasks meet certain
criteria. Thus, it is not possible to identify clearly text comprehension at some levels of
representation –the level of the propositional and situation models- if students are asked only
to explain the meaning of words contained in the text, as it often happens. It is necessary to
ask the students to make some kind of summary of its meaning and to use text information for
solving problems of some kind. That is, assessment tasks need to meet criteria defined by
psychological models of mental representations and processes to be valid as indicators of
ability, understanding and reasoning. This is especially important if assessment is aimed at
identifying the kinds of feedback and help that should be given to students in order to improve
their meaningful learning.
- 596 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
Therefore, with the aim of identifying strengths and shortcomings in teachers’ ideas
about what constitutes a valid indicator of causal understanding, and of the remaining abilities
being assessed eight similar studies were designed, using the same sample. However, due to
space limitations, the objective of the present paper is to present only the results of the study
related to ideas on assessment of causal understanding. Nevertheless, before presenting results,
it is necessary to answer a previous question: what criteria should be used to evaluate the
adequacy of teachers’ ideas on causal understanding assessment?
Theoretical Framework
Teachers of Social Sciences –Geography, History, Economics, etc.- try that their
students understand why social organisation and dynamics are different and change depending
on time and geographical area. To achieve this objective, students have to understand the
concepts created by experts in these sciences for describing and explaining the organisation and
dynamics of society. However, this objective is difficult to achieve because of the naïve ideas
that they have about the nature of History and social functioning (Voss, Wiley & Kennet, 2004),
about the nature of causality (Voss, Ciarrochi & Carretero, 2004) and about historical evidence,
etc. So, assessment tasks and criteria should warrant a) the validity of inferences on student’s
understanding of causal relations, and b) the identification of factors responsible of
understanding difficulties in order to decide the kinds of feedback and help that should be given
to students to make possible their progress. However, what does understanding of social
changes imply?
Understanding social changes implies the construction of plausible causal models able
to explain them (Leinhardt, 1994), even though such models have to be hypothetical and revisable, as they depend on available evidence and this may change. However, when and in
what degree can we say that a student has built an adequate causal model for explaining a
particular kind of social change? Let us consider, for example, the basic causal model of the
French Revolution presented in Figure 1 that students 13-15 years old are supposed to construct and understand according to the content of most Spanish history textbooks that have
been examined.
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 597 -
Figure 1. Basic causal map of “The French Revolution” for students 13 to 15 years old. (Alonso-Tapia, Asensio, Salguero & Villa, 1997)
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 598 -
What should be assessed and how –what kinds of task should be used- in order to see whether
such a model has been understood and not only memorised, and what are the reasons of students’ understanding difficulties? And what is more important, what are the tasks that teachers use for this purpose? In order to answer these questions it is worth considering the conditions supporting the building of causal models.
The adequate construction of a causal model able to explain a social change implies, in the
first place, knowing the facts and factors that were responsible of it, but this knowledge is not
enough. It is also necessary to understand the reason because of which a factor may have had a
causal effect on this change. Such understanding implies at least the five processes described
next and summarized in Figure 2
Figure 2: Facets of causal understanding and basic questions and tasks for assessing them.
1) Students must understand the concepts used for describing the different facts and
phenomena that change or contribute to the change. However, what does it imply to understand
a concept? According to Dole and Sinatra (1998), psychologists recognise that conceptual understanding -acquiring or organising conceptual knowledge- implies constructing some sort of
mental representation in the form of scripts, frames, schemata, beliefs, etc. (Anderson & Pearson, 1984; Rumelhart & Ortony, 1977: Shank & Abelson, 1977). Nevertheless, it is important to
point out that that this construction implies two different processes.
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 599 -
Jesús Alonso-Tapia et al.
In the first place, conceptual understanding implies concept formation, the construction
of categorisation rules by which: “to render discriminably different things equivalent, to group
the objects, events and people around us into classes and to respond to them in terms of their
class membership rather than their uniqueness” (Bruner, Goodnow & Austin, 1956, p.1). According to most authors (Ashby & Maddox, 2005; Markman & Ross, 2003; Murphy, 2002;
Smith & Medin, 1981), it is not necessary that the mental representation of this rule includes all
attributes of the concept. It may by an exemplar or a prototype allowing the categorisation of
phenomena on the base of their similarity to structural, functional or relational characteristics of
it. Therefore, understanding a concept implies the building of a representation allowing such
categorisation.
In the second place, conceptual understanding implies concept identification, the association of the rule underlying a verbal term to that term, an association that requires prior formation of the concept. Sometimes students may have formed a particular concept, as can be inferred from their reactions to exemplars and non-exemplars of it, but they do not know the verbal label. For example, students may react to the listening or reading of pronouns by looking for
their referents without associating the word “pronoun” with them. Other times, they associate a
verbal label to conceptions different from those that experts attach to this label. For instance,
this is the case with the concept of “alive”, which evolves from meaning, “to be in motion” to
“be born, to grow and to die” (Delval, 1975; Piaget, 1926).
It is not difficult to associate the correct verbal label to a concept as long as a person has
formed the correct categorisation rule. Thus, from our point of view, if assessment is aimed at
deciding which instructional aids should be given to students in order to favour conceptual
understanding, assessment activities should focus on the student’s rules for categorising phenomena. These rules become manifest in categorising or predicting behaviour and in the explanation of it, and not in remembering the definition of concepts, because a definition is a
“fact” that can be remembered, but being able to recall a fact does not warrant adequate understanding. For example, if a student is asked “what is a bourgeois?” he or she can answer
saying that is a rich person that lives in a city. However, when asked if a person that owns
large properties in the country but lives in a city and belongs to the royal court is a bourgeois,
the answer could be wrong if the categorisation rule used is incorrect.
- 600 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
2) Students must be able to categorise historical events in relation to the role they
play in social change. Historical events can be classified into different categories: there are
natural, demographic, technical, economic, social, political, military, personal, ideological
and religious events, to mention only some. These different categories of events may play
systematic roles in the organisation and change of human groups and societies unless there are
counteracting factors. For instance, the application of technical innovations to production crop rotation, fertilisers, computers, etc.- plays always a similar role: it helps to increase production rates and to decrease the workforce needed to achieve the same outcomes. If this is
the case, knowing the roles played by different categories of facts may be of great help in order to understand the present and, what is most important, to anticipate what might happen if
facts playing the same roles took place. So, student’s capacity for categorising historical
events and for applying their knowledge of the role played by the different categories of facts
to explain historical events and to infer possible courses of action and historical change can be
considered as an index of causal understanding. Again, tasks best suited for assessing this
aspect of historical causality are categorisation and prediction tasks similar to the tasks presented in Table 1 for assessing whether students understand some of the causal relations
shown in Figure 1, as well as tasks demanding explanation of causal roles played by specific
factors (Alonso-Tapia & Villa, 1999); Alonso-Tapia et al. 2004).
Table 1. Example of task and questions designed for assessing causal understanding.
(Alonso-Tapia, Asensio & López, 2000a).
______________________________________________________________________________________________
A. Imagine the situation described in Box 1. There are four countries differing in the amount of crops they harvest over several consecutive years. In some of these
countries there are groups of people that monopolise and retain most of the crop, whereas in other countries there are no such groups. (The situation of some of
these countries is similar to the French situation just before the French Revolution (FR)) Thus, taking into account this fact, answer the following questions.
There are people
Box 1
a.1.
that monopolise
and retain crops
Annual crop
First Year
Second Year
Third Year
Country A
Yes
Scarce
Average
Scarce
Country B
Yes
Abundant
Scarce
Scarce
Country C
No
Abundant
Scarce
Scarce
Country D
No
Scarce
Average
Average
In which of the countries will agricultural prices rise most? ___ Why do you think this?
How does the crop affect prices? ___ Why?
How does the monopolisation and retaining of crops affect prices? ___ Why?
a.2.
In which of the countries will peasants to become impoverished more likely? ___ Why?
a.3.
In which of the countries will urban middle and poor classes become discontented? ____ Why?
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 601 -
Jesús Alonso-Tapia et al.
B. Imagine five countries each one equal to the other in everything except the characteristics described in Box 2. Answer the following questions. (Taking into account
what you have learned during the study of the FR)
Box 2
b.1
Country
A
B
C
D
E
It has lost its colonies
Yes
No
Yes
Yes
Yes
Other countries sell at lower prices
Yes
Yes
No
Yes
No
Crops have being worsening
Yes
Yes
No
No
Yes
In which country is trade more likely to diminish? ___ Why do you think this?
C. Imagine five countries each one equal to the other in every thing except the characteristics described in Box 3. Answer the following questions.
(Taking into account what you have learned during the study of the FR)
c.1
Box 3
Country A
Country B
Country C
Country D
Country E
Trade with other countries
Flourishing
Diminishing
Diminishing
Flourishing
Diminishing
Prices
Rise
Rise
Rise
Established
Rise
Taxes
Rise
Rise
Established
Rise
Rise
Rights and duties of social groups
Different
Different
Different
Equal
Different
In which country would be more likely that some group was interested in changing the way it was governed? _____ Why do you think
this?
c.2
Why not each one of the other countries?
D. Imagine three countries whose economy is bad –prices rise, there are many people almost starving, etc.- and where taxes are high due to state expenditure
and debts, as in the epoch of the French Revolution. Imagine also that citizens of each one of these countries differ in their way of thinking, being the
dominant ideas in each country the following:
Country A
Power must be hold by only one person. He/she must legislate, has to be able to force law accomplishment and, if this is not the case, has to
be able to punish those being guilty.
Country B
Goods belong to everybody. Let us organise production collectively and give them to each person according to his/her needs.
Country C
The right to private property is fundamental. Therefore, every government has the obligation of making it to be respected.
d.1. Would it be likely or not that in any of these countries the way of government changed as it happened in the epoch of the French Revolution?
If such a change took place, would it be similar or not to that of the French Revolution? ______. Why do you think in this way?
3) Students must understand the interactions between different factors in producing
social changes. When students understand the conditions that help or obstruct the appearance
of a particular effect, their comprehension of historical causality increases. However, understanding such interactions is a complex process. First, students’ implicit ideas about concepts
such as inquiry, evidence, explanation and cause change and influence the interactions that
- 602 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
children of different ages are able to understand and built (Lee, Dickinson & Ashby, 1996).
Second, understanding interactions implies to know not only that two or more factors produce
an effect when combined, but also the nature of such combination: whether it is necessary and
sufficient, necessary but not sufficient, sufficient but not necessary or whether it is only a facilitating factor, concepts on which they have intuitive ideas not always adequate (Voss, Ciarrochi & Carretero, 2004). For example, an increase of taxes can exert a causal role in a radical
change like the French revolution. However, is it a necessary factor, a sufficient one or both?
Or does it only facilitate an effect caused by other factors? ¿How does the increase of taxes
interact with other factors in causing a revolution? When trying to answer questions like
these, students need to take into account conditions whose presence or absence affect the validity of inferences about causality in a process similar to what natural scientists call “control
of variables”. (Pozo & Carretero, 1989). For this reason, in order to assess causal understanding it is important the use of tasks showing whether students consider that the causal factors
included their explanations of historical and social changes are sufficient or necessary, etc.
4) Students must be able to transcend the past significance of a fact. “Transcend” in
this context means being able to understand three things: 1) A fact belongs to a given category
of factors and interacts in different ways with other factors to produce a given effect; 2) if
such a fact happened today in similar conditions, it would have analogous effects; and 3)
these effects could be achieved or avoided acting on conditions favouring the appearance or
the neutralisation of the fact considered. For example, if under certain conditions tax increase
to support military expenses can contribute to the debilitation and disappearance of an empire
like the Roman, a country need peace politics in order to avoid the need of having a big army
and so, to increase taxes up to an unbearable level. Obviously, if tax increase happens in conditions different from those that took place at the end of the Roman Empire, the effects could
be also different. Nevertheless, it is important to know the general applicability of this and
others historical lessons to avoid their negative consequences.
5) Students must be able to see that models of historical and social organisation and
change are the result of interpretations more or less supported by evidence, and that these
interpretations are subjected to change if evidence or its interpretation change. The achieving
of this objective is difficult because students are usually taught to accept without questioning
the information given by teachers and textbooks (Gardner, 1991; McKeown & Beck, 1994).
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 603 -
Jesús Alonso-Tapia et al.
So, it is important to assess whether students understand the subjectivity of historical interpretations in order to give them the adequate correcting help if necessary.
In summary, understanding social organisation, functioning and change is not an all or
none problem. Facts are first known and then gradually understood as new information is
gathered and integrated with what is already known (Alonso-Tapia, 2002; Dole & Sinatra,
1998; Limón, 2002; Pozo & Rodrigo, 2001; Rodrigo & Correa, 2001). This process, however,
may give place to misunderstandings and wrong causal models depending on students’ previous knowledge, on the focus and process of learning activities and on assessment planning
and tasks. So, it is important to know the specific factors that hinder comprehension.
Given that understanding social organisation and change in different places and at
different times depends on the five processes just described, what kinds of data can be
considered as valid criteria of understanding? According to the above ideas, understanding
will not become manifest in the reminding of facts or of the “listened” explanations of their
causes and effects, though this is the criteria most often used by teachers (Villa & AlonsoTapia, 1996). On the contrary, students will show their causal understanding when they are
able a) to categorise historical facts, b) to predict the most probable outcomes of a factor in a
given situation according to the mental models supported by historical or social evidence,
though hypothetical en revisable, c) to build narrative explanations -not previously known -of
groups of given facts, d) to explain what would have most likely happened if a given factor
would have not been present, and e) to deduce the line of action probably most adequate in a
present situation in the light of effects that alternatives lines of action have had in the past.
However, what are teachers’ ideas about how causal understanding should best be assessed in
order to identify students’ difficulties and to decide how to help them? To answer this question is
the objective of the present study.
Method
Participants
68 subjects, 38 Spanish Secondary and High School teachers (17 men and 21 women,
mean age 40.39) and 30 Brazilian teachers of the same levels (15 men and 15 women with
- 604 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
ages lower than 40 in all cases) that volunteered to participate in the study after being asked to
do it, formed the sample of the study.
Materials
All the teachers were given the following task for get evidence about their ideas on
how causal understanding should be assessed:
According to our curriculum objectives, students must understand different
kinds of causal relations between historical events –simple, multiple, necessary, sufficient, facilitating causes, etc.—in order to reach a plausible
causal explanation of them. With this assumption: What kinds of task or
question would you give to your students for assessing causal understanding
in relation to the French Revolution?
As can be seen, the question was left open in order to avoid suggestions about how
causal thinking in social sciences should best be assessed. According to the nature of causal
understanding above described, teachers should ask questions demanding not “rote learning”
of historical and social facts, but the application of mental models to categorising social
causes and effects, to explaining the role played by particular factors, to predict outcomes
most likely to occur given different contexts –demographic, economic, political, technical,
religious, etc.- and to apply such models to decide the lines of action more adequate nowadays given also particular sets of conditions. However, in order to create a minimum common
context for all the teachers, we asked them to answer the question in relation to the French
Revolution.
Procedure
Teachers’ answers to each question were coded using seven categories. Six categories
were derived from psychological ideas on the nature of conceptual and causal understanding
above described. A last category was added in order to include the tasks and questions proposed by teachers that could be answered without causal understanding. No other category
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 605 -
Jesús Alonso-Tapia et al.
was needed. Then, questions proposed for assessing causal understanding were coded in the
categories shown in Table 2.
Table 2. Categories used for coding questions and tasks for assessing students’ causal understanding,
and examples of each category.
1. Questions aimed at testing whether the students were able to differentiate causes as a function of:
a)
Their nature: natural, demographic, technical, economic, social, political, military, etc.
● (In a given a text on the French Revolution) Point out the causes and classify them as economic, social,
political and religious ones.
b)
The kind of causal relation: necessary, sufficient, facilitating, primary, secondary, structural, etc.
● Did the situation in France between 1785 and 1788 make the Revolution unavoidable?
● (In a given text on the French Revolution) Tell whether the causes are principal or secondary and explain
why you think they are like that
● Which of the following causes are structural (E), and which are temporary (T)? 1) Rising of bread price. 2)
The bourgeoisie looks for the political power. 3) Economic stagnation of the Ancient Regime. 4) Rising of
national expenditure because of the war with the American colonies.
c)
The temporal distance between cause and effect: remote, close, etc.
●
Define what are direct and indirect causes, and give examples of each kind of cause related to
the French Revolution.
2. Questions aimed at testing whether students’ remembered the causes and consequences of historical or
social events.
●
Describe the effects of the USA independence on the French Revolution.
●
What were the philosophical, political and economic causes that favoured the French
Revolution
●
Why did the French Revolution happen?
●
What were the consequences of the General States meeting?
●
Was there any relation between the rice of wheat prices and the burst of the French Revolution?
3. Questions aimed at testing whether students’ were able to elaborate a scheme or diagram making explicit
the causal relations between different events.
● Draw a diagram of the relationships between the different causes of the French Revolution and of three
effects on each of the three social states.
● Draw a diagram connecting with arrows the causes and consequences of the French Revolution.
● Explain by means of a conceptual map the causes of the French Revolution.
4. Questions aimed at testing whether the students understand the causal role played by particular factors.
●
Suppose there had been enough food. Would the Revolution had happen? Explain your answer
●
Suppose nobles had give way. Would the Revolution have burst? Explain your answer
●
- 606 -
Could a rise of taxes be the cause a revolution? Explain your answer.
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
5. Questions aimed at testing whether students’ perceived that factors of a particular kind tend to produce
consistently a given effect.
● Imagine four countries with the following characteristics (Students are given pairs of characteristics that
are similar or partially or totally opposed to the characteristics of France just before the French
Revolution). Which of them would likely have a revolution similar to the French? ¿Why?
6.
Questions aimed at assessing whether students perceive the subjectivity of causal interpretations.
● Suppose that tour textbook said that French Revolution was supported mainly by the bourgeoisie, but
your teacher said that there is evidence that nobles were as supporters as bourgeois. Which of the two
points of view do you think is correct? Why do you think so?
7. Questions demanding the remembering of facts without asking to do any reference to causes or consequences
●
Describe how people lived, the political system and the cultural values.
●
Analyse the social pyramid of the French Revolution epoch.
Answers were coded independently by at least two persons, reaching initially a high degree of
interrater agreement (more than 90%). Where discrepancies arouse, they were discussed until
a final decision on categorization was reached. Then, the proportion of teachers suggesting
tasks falling in each category was calculated and analysed using the binomial test, showing
specific strengths and shortcomings of teachers’ ideas on what can be considered valid indicators of causal understanding. Moreover, in order to test for differences of use between categories and to see whether assessment strengths and weaknesses were a problem common to
Spanish and Brazilian teachers for each category, chi-square analyses were carried out.
Results
According to criteria of causal understanding previously established, teachers should
have proposed a set of questions for categories related to each one of the comprehension
levels, and not the remembering of historical and social events. This fact implies that the use
of the first seven categories should have been approached 100% -though the use of the
seventh would have been correct or not depending of the degree in which the previous levels
had been also assessed- and that of the eight, near 0%. Nevertheless, the analysis of the
degree in which each category was used showed that the situation is far from this ideal. In
fact, the observed proportions in categories 1 to 7 are significantly lower than the desired
value 0.99 (p<0,0001) and the proportion in category 7 is significantly higher than the desired
value 0,01 (p<.0001). Data on causal understanding assessment are shown in Table 3.
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 607 -
Jesús Alonso-Tapia et al.
Table 3. Percentages of teachers using each category of questions for assessing students’ causal
understanding ,and differences between Spanish and Brazilian teachers.
% of teachers using them
CATEGORIES OF QUESTION
Total
Spain
Brazil
N: 68
N: 38
N: 30
Differences
χ2
F
%
F
%
F
%
1
1,4*
1
2,6
0
0,0
.80 .37
2
2.9*
1
2,6
1
3.3
.03 .86
1
1,4*
1
2,6
0
0.0
.80 .37
39 57,3*
22
57,9
17
56,6
.01 .92
3*
1
2,6
1
3.3
.03 .86
2,9*
1
2,6
1
3.3
.03 .86
0*
0
0
0
0
-
-
0
0*
0
0
0
0
-
-
8
12**
0
0
8
26.6
p
1. Questions aimed at testing whether the students were able to
differentiate causes as a function of:
a) Their nature: natural, demographic, technical, economic,
social, military, etc.
b) The kind of causal relation: necessary, sufficient, facilitating,
structural, etc.
c) The temporal distance between cause and effect: remote,
close, etc.
2. Questions aimed at testing whether students’ remembered the
causes and consequences of historical or social events.
3. Questions aimed at testing whether students’ were able to
elaborate a scheme or diagram making explicit the causal 2
relations between different events.
4. Questions aimed at testing whether the students understand the
causal role played by particular factors.
2
5. Questions aimed at testing whether students’ perceived that
factors of a particular kind tend to produce consistently a given 0
effect.
6. Questions aimed at assessing whether students perceive the
subjectivity of causal interpretations.
7. Questions demanding the remembering of facts without asking to do
any reference to causes or consequences
*
11.5 .001
Value significantly different from .99. p<0.0001
**
Value significantly different from .01. p<0.0001
The analysis of these results shows the following facts:
a)
First, there are big differences in the degree in which teachers have proposed the
several kinds of question for assessing causal understanding (from 0% to 57.3%,
mean 8.98%). Most categories are almost empty. The main exception is the category
“questions aimed at testing whether students’ remembered the causes and
- 608 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
consequences of historical or social events”. These questions are suggested as valid
criteria by 58% of teachers. This datum is coherent with results found by Villa and
Alonso-Tapia (1996), in which 68% of questions actually asked by Secondary and
High School teachers demanded from the students the recall of facts and of
explanations given by their teachers. The review of Black and William (1998) has
found similar results. However, remembering does not imply necessarily
understanding, as Voss, Wiley and Kennet (2004) have shown after examining the
role of research methodology in the conclusions reached on students’ understanding.
A student can remember without understanding what is remembered, or can
understand without being able to recall.
b)
Second, the category “Questions demanding the remembering of facts without asking to do any reference to causes or consequences” is the second category most frequently suggested, but only by Brazilian teachers (26%), as it is the only category in
which differences between the two countries have been significant. This fact, as the
previous one is also negative because it implies that many teachers consider as valid
indicators of causal understanding the answers to questions that are not adequate for
assessing it.
c)
Third, as has already been said, the percentage of teachers that suggests questions or
task of the remaining categories is very low.
ƒ
There are almost no questions aimed at testing whether the students were able to
differentiate causes as a function of there nature -natural, demographic, technical,
economic, social, political, military, etc.- (0.4%). Teachers may make explicit the
kind of cause they are referring to if the “standard” or most accepted explanation
demands such reference. However, the lack of a systematic assessment of the
capacity of distinguishing between each kind of cause makes difficult to know
whether students are able to consider the potential causal role of these different
kinds of factors.
ƒ
The use of counter factual questions of the type: “Would B have happened if A
had not occurred?” make ease to assess whether the student consider or not A as a
necessary cause. In a similar way, the use of questions of the type: “Do you think
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 609 -
Jesús Alonso-Tapia et al.
that each time A occurs, B occurs too?” make it easy to assess the sufficiency of a
cause. However, in spite of the importance of distinguish between both kinds of
causal relation -necessary, sufficient, etc.- (Voss, Ciarrochi & Carretero, 2004)
and that it is very easy to assess them, only 2% of teachers pointed to the
importance of assessing these aspects of causal understanding.
ƒ
Only 3% of teachers suggested to ask questions aimed at testing whether students’
were able to elaborate a scheme or diagram making explicit the causal relations
between different events and to explain it. One could think that a percentage so
low is due to the similarity between this task and that of remembering a causal
explanation. However, it is not necessarily so, such as Lee et al. (1996) suggest.
These authors gave the students several possible causes written in boxes to explain
a fact -the invasion of Great Britain by the Roman Empire-, also written in a box.
Students had to draw arrows starting in box representing causes and ending in the
fact to be explained. They could also draw arrows connecting causal factors
between them, as if they formed a causal chain. Finally they had to explain their
options. This task implies cognitive processes different from remembering because
the student is confronted with the need to justify his or her options.
ƒ
Again, only one Spanish teacher suggested the need to assess whether the students
understand the kind of causal role played by specific factors using questions
similar to that exemplified in Table 3, category 4, and no teacher has suggested the
use of questions aimed at testing whether students’ perceived that factors of a
particular kind tend to produce consistently a given effect (category 5). However,
as it happened with questions commented in the previous paragraph, these
questions are especially adequate for assessing causal understanding because the
students have to explain their answers, what obliged them to make explicit the
mental representation of the causal relation explored.
ƒ
Finally, no teacher suggested assessing whether students understand that causal
explanations are “interpretations”, that is, that they are subjective and might
change.
- 610 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
Discussion and Conclusions
Before entering the discussion, it is important to note that although the sample was
small, data from both countries were very similar. Moreover, these results are similar to those
found in the pre-test data of an intervention study using a different sample (Alonso-Tapia,
2004). These facts convey great relevance to our results as a pilot diagnostic of social science
teachers’ strengths and limitations in relation to assessment.
It was pointed out at the beginning of this paper that, in order to know the kind of
feedback and help that teachers should give to their students for improving causal understanding, an adequate assessment of facets of causal understanding was needed. Previous studies
(Villa & Alonso-Tapia, 1996) had shown that social science teachers assessed rote-learning
most of the time. This practice is inadequate because remembering does not necessarily imply
understanding. The question was then whether teachers’ assessment practices were inadequate
for practical reasons or because their ideas about how to assess conceptual and causal understanding were inadequate. To answer this question the present study has tried to determine
whether teachers’ ideas on how to assess conceptual and causal understanding have the characteristics that, according to psychology, allow a valid assessment of these processes.
However, the results have shown that Spanish and Brazilian teachers’ ideas on causal
understanding assessment are not adequate. They consider that being able to remember the
causal models described in textbooks or explained in class provides enough evidence of
understanding and, with the exception of one teacher or two, they do not mention other kinds
of question or task. A quite acceptable situation would have been that each teacher had
suggested questions for assessing the different aspects of causal understanding described in
the theoretical section, but in relation to the basic model of the French Revolution presented
in Figure 1. They could have suggested, for example, the assessment of concepts implied in
causal explanations trough categorization tasks, and the use of tasks and questions of the type
included in Table 1. As can be seen, they are hypothetical situations designed to assess
conceptual understanding of some causal relations included in Figure 1. Each situation is
followed by questions asking for inferences about what might happen in relation to prices,
impoverishment of peasants, problems of city dwellers, etc. Students have to justify their answers,
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 611 -
Jesús Alonso-Tapia et al.
so that the mental representations determining them can be detected. Moreover, as it has been
mentioned above, teachers could have suggested also counter factual questions of the type:
“Would have happened B if A did not had occurred?” or “Do you think that each time A
occurs, B occurs too?” to assess the necessity or the sufficiency of a cause, etc. Of course, all
these kinds of questions can be asked when giving traditional exams or in the context of new
ways of collecting evidence –performance tasks, learning journals, portfolios, etc.- (AlonsoTapia, 2002; Birembaum, 2003).
In summary, on one side this study has shown specific shortcomings in teachers’ ideas
about how causal understanding should best be assessed. Shortcomings indicate that it is
necessary to promote a reflection from the part of teachers on the degree of adequacy of the
tasks that they consider as valid criteria for assessing conceptual and causal understanding
and to suggest them alternative assessment models, as it has been already proposed (AlonsoTapia, 1997, 2002; Birembaum et. al., 2006; Davies & LeMahieu, 2003). On the other side,
this study has shown also the usefulness of exploring teacher’s thinking related to assessment
by means of open questions demanding the design of real assessment tasks, and of using the
theoretical models provided by psychology bout each one of the processes that are the object
of assessment for evaluating the adequacy of these tasks. If the use of this methodology
showed also the inadequacy of tasks used for assessing the degree of acquisition of other
cognitive abilities that social sciences teaching should favour, such as those implied in
understanding the information included in different kind of documents –texts, tables, graphs,
objects, etc., teacher training should offer also alternate models for assessing these cognitive
abilities. Nevertheless, this is a task for forthcoming papers (Alonso-Tapia and al., 2006).
- 612 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
References
Alonso-Tapia, J. (Dir.) (1997). Evaluación del conocimiento y su adquisición. Vol. 1: Ciencias Sociales. [Assessment of knowledge and its acquisition. Vol.1: Social Sciences].
Madrid: Ministerio de Educación y Ciencia.
Alonso-Tapia, J. (2002). Knowledge assessment and conceptual understanding. En M. Limón &
L. Mason (Eds.), Reframing the processes of conceptual change (pp. 389-413).
Dordretch: Kluwer
Alonso-Tapia, J., Asensio, F., López, I. & Carriedo, N. (2004). Evaluación del conocimiento y
formación del profesorado: Diseño, evaluación y valoración inicial de un programa de
Formación del Profesorado de Secundaria en evaluación de conocimientos y capacidades cognitivas [Knowledge assessment and teacher education: Design, assessment and
pilot evaluation of a training program addressed to social science teachers on assessment
of knowledge and cognitive abilities]. In Ministerio de Educación, Cultura y Deporte
(Ed.) Premios Nacionales de Investigación Educativa 2002. [National Prices of Educational Research 2002] Madrid: Ministerio de Educación y Cultura.
Alonso-Tapia, J., Asensio, F., López, I., Carriedo, N. & Rychecki, F. (2006). How cognitive
abilities implied in text, table and graph understanding should best be assessed?
Strengths and shortcomings of Social Science teachers’ ideas on this point. Unpublished manuscript, submitted for publication. Madrid: Universidad Autónoma.
Alonso-Tapia, J. & Pardo, A. (2006). Assessment of learning environment motivational quality from the point of view of secondary and high school learners. Learning & Instruction, 16, 295-309.
Alonso-Tapia, J. & Villa, J.L. (1999). How can historical understanding could best be assessed?
Use of prediction tasks to assess how students understand the role of causal factors that
produce historical events. European Journal of Psychology of Education, v. 14, 3, 339358
Anderson, R. C. & Pearson, P.D. (1984). A schema theoretic view of basic processes in reading comprehension. In R. Barr, M.L. Kamil & P. Mosenthal (Eds.), Handbook of reading research. (pp. 255-291). New York: Longman.
Ashby, F.G. & Maddox, W.T. (2005). Human category learning. Annual Review of Psychology,
56, 149-178.
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 613 -
Jesús Alonso-Tapia et al.
Birembaum, M. (2003). New insights into learning and teaching and their implications for
assessment. In M. Segers, F. Dochy, & E. Cascallar (2003). Optimizing new modes of
assessment: In search for qualities and standards. (pp. 13-36). Boston: Kluwer.
Birembaum, M., Breuer, K. Cascallar, E., Dochy, F., Ridgway, J., Wiesemes, R. & West, A.
(2006). A learning integrated assessment system. Educational Research Review, 1 (1),
61-67.
Black, P. & William, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5, 7-74.
Bruner, J.S., Goodnow, J.J. & Austin, G.A. (1956). A study of thinking. New York: Wiley
Crooks, T.J. (1988). The impact of classroom evaluation practice on students. Review of
Educational Research, 58, 348-381.
Davies, A. & LeMahieu, P. (2003). Assessment for learning: reconsidering portfolios and
research evidence. In M. Segers, F. Dochy, & E. Cascallar (2003). Optimizing new
modes of assessment: In search for qualities and standards. (pp. 141-169). Boston:
Kluwer.
Delval, J. (1975). El animismo y el pensamiento infantil. Madrid: Siglo XXI.
Dochy, G. (2005). “Learning lasting for life” and “assessment”: How far did we progress?
Presidential address, EARLI-2005. Nicosia, Cyprus.
Dole, J. A. & Sinatra, G. M. (1998). Reconceptualizing change in the cognitive construction
of knowledge. Educational Psychologist, 33 (2/3), 109-128.
Gardner, H. (1991). The unschooled mind: How children think and how schools should teach.
New York: Basic Books.
Lee, P., Dickinson, A. & Ashby, R. (1996). “There were no facts in those days”. Children
ideas about historical explanation. In M. Hughes (Ed.), Teaching and learning in
changing times. Oxford: Blackwell.
Leinhardt, G. (1994). History: A time to be mindful. En G. Leinhardt, I.L. Beck & C. Stainton,
(Eds.), Teaching and learning in history. (pp. 209-255). Hillsdale, NJ: Lawrence
Erlbaum.
Limón, M. (2002). Conceptual change in History. En M. Limón & L. Mason (Eds.), Reframing
the processes of conceptual change (pp. 259-289). Dordretch: Kluwer
Limón, M. & Mason, L. (Eds.) (2002). Reframing the processes of conceptual change.
Dordretch: Kluwer
- 614 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
Assessment for learning: Social science teachers’ ideas on assessment of causal understanding
Markman, A.B. & Ross, B.H. (2003). Category use and category learning. Psychological
Bulletin, 159, 592-613.
McKeown, M & Beck, I.L. (1994). Making sense of accounts of history. Why young students
don’t and how they might. In G. Leinhardt, I.L. Beck & C. Stainton (Eds.), Teaching
and learning in history. (pp. 1-27). Hillsdale, NJ: Lawrence Erlbaum.
Murphy, G.L. (2002). The book of concepts. Cambridge, MA: MIT Press.
Piaget, J. (1926). La representation du monde chez l’enfant. Paris: Alcan.
Pozo, J.I. & Carretero, M. (1989). Las explicaciones causales de expertos y novatos en historia.
En M. Carretero, J.I. Pozo & M. Asensio (Eds.), La enseñanza de las Ciencias Sociales.
(pp. 139-164). Madrid: Visor.
Pozo, J.I. & Rodrigo, M.J. (2001). Del cambio del contenido al cambio representacional en el
conocimiento conceptual. Infancia y Aprendizaje, 24 (4), 407-423.
Rodrigo, M.J. & Correa, N. (2001). Representación y procesos cognitivos. Esquemas y
procesos mentales. In C. Coll, J. Palacios & A. Marchesi (Coord.), Desarrollo
psicológico y educación. 2. Psicología de la educación escolar. (117-135). Madrid:
Alianza.
Rumelhart, D.E. & Ortony, A. (1977). The representation of knowledge in memory. En R.C.
Anderson, R.J. Spiro & W.E. Montague (Eds.), Schooling and the acquisition of
knowledge (pp. 99-135). Hillsdale, NJ: Lawrence Erlbaum.
Shank, R.C. & Abelson, R. (1977). Plans, scripts, goals and understanding. Hillsdale, NJ:
Lawrence Erlbaum.
Segers, M., Dochy, F. & Cascallar, E. (2003). Optimizing new modes of assessment: In
search for qualities and standards. Boston: Kluwer.
Smith, E.E. & Medin, D.L. (1981). Categories and concepts. Mass: Harvard University Press.
Villa, J.L. & Alonso Tapia, J. (1996). Evaluación del conocimiento: Procedimientos
utilizados por los profesores en BUP y FP. [Knowledge assessment: Procedures used
by secondary and high school teachers] In Ministerio de Educación y Ciencia (Ed.),
Premios Nacionales de Investigación e Innovación Educativa 1994 (pp. 51-78).
[National Prices of Educational Research 1994] Madrid, Ministerio de Educación y
Ciencia, CIDE.
Voss, J.F., Ciarrochi, J. & Carretero, M. (2004). La causalidad histórica: acerca de la
comprensión intuitiva de los conceptos de suficiencia y necesidad. En M. Carretero &
J.F. Voss (Eds.), Aprender y pensar la historia. (pp. 273-293). Buenos Aires:
Amorrortu.
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616
- 615 -
Jesús Alonso-Tapia et al.
Voss, J.F., Wiley, J. & Kennet, J. (2004). Las percepciones de los estudiantes acerca de la
historia y los conceptos históricos. . En M. Carretero & J.F. Voss (Eds.), Aprender y
pensar la historia. (pp. 295-329). Buenos Aires: Amorrortu.
- 616 -
Electronic Journal of Research in Educational Psychology, N.13 Vol 5 (3), 2007. ISSN: 1696-2095. pp:593-616