Projects
Although I primarily have been labeled as working in the area of metacognitive judgments and decision making, my interests span a wide spectrum within cognitive psychology. Additionally, I am interested in research questions that are both basic and applied in nature. Below are a few specific representative examples of my current lines of research.
Metacognitive monitoring, type-2 signal detection, and formula-scoring
One of the best known schemes for interpreting test responses
is a number-right scoring system. That is, students are instructed to
answer all questions on an exam, and their overall test score is
calculated by simply assigning one point for each question that is
answered correctly and zero points for incorrect responses. An
alternative to number-right scoring that is popular in educational
settings is formula-scoring. Under formula-scoring rules, students are
told that they can withhold responses (i.e., leave questions
unanswered), and the test typically is scored by giving points to
correct responses, a penalty for incorrect answers, and zero points for
withheld responses. The goal of formula-scoring, and the main reason
that supporters of this technique advocate its use over number-right
scoring, is to produce a more “pure” measure of knowledge level.
Specifically, it has been argued that guessing on an exam introduces
variance into the test scores, and formula-scoring is supposed to
reduce this variance by using a penalty to both suppress the tendency to
guess and to remove the variance that is due to any guessing that did
occur.
Because the methodology of formula-scoring requires
individuals to decide when to report and when to withhold responses, it
has the built-in assumption that people have the ability to accurately
monitor their level of knowledge. Formula-scoring also requires
individuals to combine an ability to monitor knowledge with the
parameters of the test to produce the most beneficial strategy for
maximizing their test scores. That is, the number of answers a person
should withhold on an exam to obtain the best score is not static across
all testing situations, in that it will vary according to his/her
knowledge level and the number of points awarded and removed for
correct and incorrect responses, respectively.
A general recommendation of how many questions to leave unanswered (e.g., always leave 5% of the questions unanswered) will not benefit all students. Phil Higham developed a technique for determining each individual’s optimal bias – that is, the number of questions that s/he should withhold in a given situation to attain her/his highest possible corrected score for a given testing situation. In brief, the formula for calculating optimal bias is based on measures produced using type 2 signal-detection theory (SDT), which itself provides indices of metacognitive monitoring (e.g., d') and response bias (e.g., B).
Phil and I have applied this optimal bias technique to
classroom test scores to examine whether students are able to monitor
their knowledge, and therefore achieve their maximum corrected score
under formula-scoring rules (Higham & Arnold, 2007). Overall, the
results indicated that the students were not performing at their
optimal bias across three separate exams; that is, their optimal bias
(the number of questions they should have left unanswered) was
significantly greater than their actual bias (the number of questions
they did leave unanswered).
More recently, we explored whether we could use the measure of optimal bias to train students to develop a better responding strategy and thereby maximize their test scores. The data thus far demonstrates that optimal bias may be a useful tool for training participants to better monitor their knowledge and increase their test scores (Arnold, Higham, and Martin Luengo, in prep). My current research on the strategic regulation of accuracy in test-taking situations is geared toward further exploring the issue of using optimal bias to train performance, including finding the most effective training techniques.
Beyond test-taking situations, I also am interested in using this methodology to explore metacognitive monitoring and optimal bias in other cognitive paradigms. For example, at present I am using this methodology to explore memory versus monitoring issues responsible for the own-race bias in face identification. Specifically, although there is a great deal of research focused on identification issues related to own- versus other-race faces, very little of this work has explored whether monitoring is a key factor. One of the main goals of this line of research in my lab is to directly measure the contribution of monitoring. The initial results from this type-2 SDT methodology have indicated that monitoring is an important component of the own-race bias.
Judgment and decision making issues related to missing person flyers and wanted posters
Although a great deal
of research has been conducted on eyewitness identification, the related issue
of missing person identification has largely been overlooked (and similarly, identification of wanted individuals).
Examining judgment and decision making issues for individuals from missing person flyers is essential for two main reasons. First, although numerous people are reported missing each year, the media often focuses on only a few high-profile missing persons and therefore the majority of cases are made known to the public exclusively through the use of individual missing person flyers. Second, the situations and context surrounding this type of identification are significantly different than any existing experimental paradigms.
The
Impact of Testing Procedure on Memory and Metamemory Performance
There are
many different ways to ask the same question. For example, a police officer may
show a witness a mug shot and ask the witness to rate on a 6-point scale how
likely this suspect was the perpetrator of the crime (e.g., from 1 =
‘definitely not the bank robber’ to 6 = ‘definitely the bank robber’).
Conversely, a different police officer may ask the witness first to decide
whether the suspect was the perpetrator (yes/no), followed by a confidence
rating (e.g., ‘definitely sure,’ ‘somewhat sure,’ or ‘guess’). Both versions of
the question require the witnesses to provide the same information; that is,
how confident they are that a suspect was or was not the perpetrator of a
crime. However, the different structures of the questions – in this case, a
one-step (6-point scale) versus two-step (yes/no, confidence) assessment
process – may influence the witnesses’ decision.
The above example highlights the
importance of exploring whether the structure of the questions we use in
experimental psychology has an influence on performance. The motivation for
this topic has roots in research on Remember/Know (R/K) judgments, which
typically are used to study the phenomenology of memories. More specifically,
there is evidence that one-step judgments (i.e., judging test items as R, K, or
New) lead participants to be more willing in general to claim that both studied
and new test items had been studied, compared to when the same judgments are
made in two steps (i.e., old/new judgment, followed by an R/K judgment for
‘old’ items). Methodological
issues, such as those related to using one- versus two-step R/K judgment, also
may be important for other tasks that we commonly use in our research. I have a
line of research that is directed at exploring several issues surrounding
testing procedure, and one of my current goals is to examine how we
traditionally collect confidence data in memory and metamemory experiments (Arnold, 2008).
Additionally, I am interested in looking at whether we can tease out different
information (or perhaps even make judgments “easier”) by simplifying certain
metacognitive judgments; for example, asking participants both for their
confidence that a chosen answer is correct and their confidence that they have
made the correct decision to act or not act on that chosen answer.
The forgot-it-all-along effect
My work with the forgot-it-all-along (FIA) effect began with my MSc
degree when I worked in conjunction with my supervisor, Steve Lindsay,
on a series of experiments focused on whether context at recall
influences remembering of previous recollection of an event. More
specifically, Jonathan Schooler and colleagues coined the phrase
“forgot-it-all-along” to describe cases in which women who reported recovered memories of abuse had apparently forgotten that they had
discussed the abuse with others during the time they believed they were
amnesic for it. We developed a laboratory analogue of this effect by
manipulating the context of studied homograph words on two separate
retrieval tests (cued-recall), and our results indicated that
participants more often forget prior remembering when they had been
cued to think of the items differently on the two tests. Further, we
have found a FIA effect both when we manipulated the dictionary meaning
of the homographic target words (e.g., hand - palm vs. tree - palm)
and with more subtle changes in context (e.g., He used the palm of his
hand to swat the fly vs. The fortune teller traced the lifeline on the
palm of his hand), as well as when we replace the first cued-recall task
with free recall.
Following on from our original work, we worked with researchers in the Netherlands to apply the laboratory version of FIA to a wider spectrum of participants, as well as implementing a different type of stimuli. Specifically, the traditional FIA paradigm (i.e., manipulating context of homographic target words) was run with participants who either had continuous memories of childhood sexual abuse (CSA), recovered memories of CSA, or no reported incidence of CSA (control group). A second experiment also involved testing these three types of participants in a FIA paradigm, but the context of autobiographical memories was manipulated instead of word stimuli. Overall, the results indicated that participants with continuous memories of CSA performed relative to control participants; that is, both controls and continuous CSA participants were much more likely to remember that they had previously recalled a target word or autobiographical memory if they were tested in the same context across the tests than when the context differed between recall attempts. The participants with recovered memories of CSA also demonstrated the typical FIA effect, but the size of their FIA effect was significantly larger than both controls and participants with continuous CSA memories (Geraerts et al., 2006, 2009).
Currently, I am interested in further teasing apart why some individuals demonstrate an enhanced FIA effect, and whether this is specific to the general FIA paradigm (i.e., difficulties in making retrospective memory judgments), or whether such individuals have a more general problem with metacognitive judgment and decision making (e.g., difficulties with both retrospective and prospective memory judgments).
I also am interested in the flip-side of FIA, that is, the knew-it-all-along effect (i.e., hindsight bias).
Participate in our Experiments
We are constantly running both paid experiments and credit experiments (i.e., credits for our first-year psych students). If you are interested in participating in any of our experiments, or in the experiments of other labs within the School of Psychology at Flinders University, then please visit our online research participating system: http://flinders.sona-systems.com
To join the research participation system you do need to have a Flinders email account, but both staff and student are welcome to participate. The majority of studies require that you physically come in to the research lab, although some studies are run online.
