In 2017 my Website was
migrated to the clouds and reduced in size.
Hence some links below are broken.
One thing to try if a “www” link is broken is to substitute “faculty” for
“www”
For example a broken link
http://www.trinity.edu/rjensen/Pictures.htm
can be changed to corrected link
http://faculty.trinity.edu/rjensen/Pictures.htm
However in some cases files had to be removed to reduce the size of my
Website
Contact me at rjensen@trinity.edu if
you really need to file that is missing
Metacognitive
Concerns in Designs and Evaluations of Computer Aided Education and Training:
Are We Misleading Ourselves About Measures of Success?
Bob
Jensen
Trinity University
715 Stadium Drive
San Antonio, TX 78212
Phone: 210-736-7347 Fax: 210-736-8134
email: rjensen@trinity.edu
Request for a Favor: This document is the first of a
sequence of research papers that are related to my Working
Paper 255 and Working Paper 260. All are
extensions of the Jensen and Sandlin online book. The document that follows
is a very rough draft. I would appreciate any feedback that
you can provide by email, phone, fax, or letter. I want to improve this
paper for my scheduled future workshops.
Table of Contents
Update Messages
Introduction
Metamemory and Metacognition:
The Metalevel Activities of the Brain
Making Learning More
Easy, Fun and Collaborative: Are We Taking Things Too Far in Virtual Learning Worlds?
The BAM Pedagogy at the University of
Virginia
Differences Between Traditional Group Learning and
Cooperative Learning
Warnings:
Suggestions for Future Research and Designs for Asynchronous Learning Networks
Conclusion
From Villanova University: BAM
for Managerial Accounting Education
The Where's My Professor Game at Brigham Young University
Accounting Professors in Support
of Online Testing That, Among Other Things, Reduces Cheating
Acknowledgement of Paula Hertel
Appendix 1
Why Aren't Stories Good Food?
Email Messages About Evaluation Criteria and Processes
Appendix 2
The Differences Between Traditional Group Learning and Cooperative Learning
Appendix 3
The Emperor's Naked as He Can Be
Appendix 4
Update Message from Robert Bjork
MIT
Encyclopedia of the Cognitive Sciences (shown
in a new window)
Appendix 5
"
Assessing the Impact of Instructional Technology on Student
Achievement," by Lorraine Sherry, Shelley Billig, Daniel Jesse, and Deborah
Watson-Acosta.
Appendix 6
February 2003 Updates on Accounting Education Pedagogy
Appendix 7
How the Brain Deals With Information Overload
Appendix 8
The Importance of Paying Attention for Longer Periods of Time
Appendix 9
How to Train the Aging Brain
Appendix 10
That Placebo Effect in
Research: Dan Ariely on Tennis Shoes and Toilet Paper
Appendix 11
Computer Trained Yet
Deeply Intuitive
Appendix 12
College Degrees Without Instructors
Appendix 13 Updates
Yeah Right!
"Forget What You Know About Good Study Habits," by Guy Spier & Peter
Rothman, Simoleon Sense, September 8, 2010 ---
http://www.simoleonsense.com/forget-what-you-know-about-good-study-habits/
Update Messages
A study says smooth-talking professors can lull students into thinking
they've learned more than they actually have -- potentially at the expense of
active learning.---
https://www.insidehighered.com/news/2019/09/09/study-how-smooth-talking-professors-can-lull-students-thinking-theyve-learned-more?utm_source=Inside+Higher+Ed&utm_campaign=f6c035d588-DNU_2019_COPY_01&utm_medium=email&utm_term=0_1fcbc04421-f6c035d588-197565045&mc_cid=f6c035d588&mc_eid=1e78f7c952
Active Learning Works But Student’s Don’t Like It ---
https://www.pnas.org/content/early/2019/09/03/1821936116
Bob Jensen's threads on tools that can improve both lecture and active
learning ---
http://faculty.trinity.edu/rjensen/000aaa/thetools.htm
Education is an admirable thing, but it is well to
remember from time to time that nothing that is worth learning can be taught.
Oscar Wilde
"The Objective of Education is Learning, Not Teaching (audio version
available)," University of Pennsylvania's Knowledge@Wharton, August 20, 2008
---
http://knowledge.wharton.upenn.edu/article.cfm;jsessionid=9a30b5674a8d333e4d18?articleid=2032
If everyone is thinking alike, then somebody isn't
thinking.
George S. Patton
"The Secret to Learning Anything: Albert Einstein's Advice to His Son,"
by Maria Popova, Brain Pickings, June ---
http://www.brainpickings.org/index.php/2013/06/14/einstein-letter-to-son/
Metacognition ---
https://en.wikipedia.org/wiki/Metacognition
Learning Intentionally and the Metacognitive Task
SSRN, 2016
https://outatpapers.ssrn.com/sol3/papers.cfm?abstract_id=2831242
Journal of Legal Education, Volume 65, Number 4, Summer 2016 Hofstra
Univ. Legal Studies Research Paper No. 2016-08
Authors
Patti Alleva University of North Dakota - School of Law
Jennifer A. Gundlach Hofstra University - Maurice A. Deane School of Law
Abstract
This article serves both to frame The Pedagogy of Procedure symposium it
introduces and to itself explore the importance of metacognition and
learning about learning to legal education and lawyering. The authors begin
by suggesting why Civil Procedure doctrine is so challenging to teach and
learn, noting how the symposium pieces help to tackle those challenges. They
then join the growing number of law professors who advocate that learning
how to learn deserves greater attention in the law school curriculum,
suggesting that law schools should do more to demonstrate respect for the
process of learning as an end in itself. They especially extol and explain
the use of metacognitive strategies to help students develop greater
self-sufficiency and proficiency in confronting learning challenges of any
kind, Civil Procedure or otherwise. They highlight metacognition because of
its importance to self-regulated learning and its benefits for professional
development. To do so, they draw upon the literature in this area, from law
faculty and from faculty in other disciplines, to create a helpful
mini-primer-plus for use in Civil Procedure and other doctrinal courses.
They close with suggestions for how law schools can show more institutional
respect for learning as a subject worthy of independent attention.
Reading Apprenticeship at WestEd: Downloadable
Resources (metacognition) ---
http://readingapprenticeship.org/publications/downloadable-resources
Metacognitive Bookmark ---
http://readingapprenticeship.org/wp-content/uploads/2014/01/RFU-ch-4-metacog-bookmark.pdf
From Emory University
Study Skills Tip Sheets & Advice ---
http://www.college.emory.edu/home/academic/learning/studyskillsconsultations/tips.html
Advice From Students
Study Skills Tip Sheets
Links to Academic Resources
Bob Jensen's threads on Tools and Tricks of the Trade ---
http://faculty.trinity.edu/rjensen/000aaa/thetools.htm
"Why Floundering Is Good Trying and failing leads to faster learning,"
by Annie Murphy Paul, Psychology Today, July 11, 2012 ---
http://www.psychologytoday.com/blog/how-be-brilliant/201207/why-floundering-is-good
Call it the “learning paradox”: the more you
struggle and even fail while you’re trying to master new information, the
better you’re likely to recall and apply that information later.
The learning paradox is at the heart of “productive
failure,” a phenomenon identified by Manu Kapur, a researcher at the
Learning Sciences Lab at the National Institute of Education of Singapore.
Kapur points out that while the model adopted by many teachers and employers
when introducing others to new knowledge — providing lots of structure and
guidance early on, until the students or workers show that they can do it on
their own — makes intuitive sense, it may not be the best way to promote
learning. Rather, it’s better to let the neophytes wrestle with the material
on their own for a while, refraining from giving them any assistance at the
start. In a paper published earlier this year in the Journal of the Learning
Sciences, Kapur and a co-author, Katerine Bielaczyc, applied the principle
of productive failure to mathematical problem solving in three schools in
Singapore.
With one group of students, the teacher provided
strong “scaffolding” — instructional support — and feedback. With the
teacher’s help, these pupils were able to find the answers to their set of
problems. Meanwhile, a second group was directed to solve the same problems
by collaborating with one another, absent any prompts from their instructor.
These students weren’t able to complete the problems correctly. But in the
course of trying to do so, they generated a lot of ideas about the nature of
the problems and about what potential solutions would look like. And when
the two groups were tested on what they’d learned, the second group
“significantly outperformed” the first.
The apparent struggles of the floundering group
have what Kapur calls a “hidden efficacy”: they lead people to understand
the deep structure of problems, not simply their correct solutions. When
these students encounter a new problem of the same type on a test, they’re
able to transfer the knowledge they’ve gathered more effectively than those
who were the passive recipients of someone else’s expertise.
In the real world, problems rarely come neatly
packaged, so being able to discern their deep structure is key. But, Kapur
notes, none of us like to fail, no matter how often Silicon Valley
entrepreneurs praise the salutary effects of an idea that flops or a
start-up that crashes and burns. So, he says, we need to “design for
productive failure” by building it into the learning process. Kapur has
identified three conditions that promote this kind of beneficial struggle.
First, choose problems to work on that “challenge but do not frustrate.”
Second, provide learners with opportunities to explain and elaborate on what
they’re doing. Third, give learners the chance to compare and contrast good
and bad solutions to the problems. And to those students and workers who
protest this tough-love teaching style: you’ll thank me later.
Competency-Based Programs (where instructors do not assign the grades) Can
Work Well But Do Not Always Work Well
A Research Report
"Competency-Based Degree Programs in the U.S. Postsecondary Credentials for
Measurable Student Learning and Performance," Council on Adult and Experiential
Learning," 2012 ---
http://www.cael.org/pdfs/2012_CompetencyBasedPrograms
Executive Summary
As our economy evolves, there is growing recognition of the importance of an
educated workforce. A key challenge is how to help more people, particularly
adults, succeed at the postsecondary level and earn degrees. However,
promoting degree completion is not our only challenge. Today our higher
education system is facing a crisis regarding its perceived quality. One
model for improving quality is competency-based education, in which an
institution clearly defines the specific competencies expected of its
graduates. This paper examines the current state of competency-based
postsecondary education in the U.S., profiling the various types of
competency-based, or competency-focused, models that currently exist, the
extent to which these programs assess for student competencies or learning
outcomes, and the extent to which these programs operate outside of a
credit-based system. These programs can help inform other institutions
interested in developing a stronger focus on competencies, whether by
demonstrating the possibilities of high quality programs or by facilitating
the recognition of learning.
Jensen Comment
The good news is that competency-based grades virtually put an end to games
played by students to influence their grades from their instructors. Instead
they may be more demanding on their instructors to do a better job on content
rather than being their buddies. Competency-based grading goes a long way to
leveling the playing field.
However, a competency-based system can be dysfunctional to motivation and
self-esteem. One of my old girl friends at the University of Denver was called
in by her physical chemistry professor who made a deal with her. If she would
change her major from chemistry he agreed to give her a C grade. I honestly
think an F grade would've discouraged her to a point where she dropped out of
college. Instead she changed to DU's nursing school and flourished with a 3.3
gpa. Purportedly she became an outstanding nurse in a long and very satisfying
career that didn't require much aptitude for physical chemistry. For some reason
she was better in organic chemistry.
I can't imagine teaching a case course in the Harvard Business School where
the course grades are entirely based on a final examination that depends zero
upon what the course instructor feels was "class participation." There's not
much incentive to participate in class discussions if the those discussions
impact some way upon grades and instructor evaluations (such as evaluations for
graduate school and employment).
Much of what is learned in a course or an entire college curriculum cannot be
measured in test grades and term paper grading (where the readers of the term
papers are not the instructors).
In spite of all the worries about competency-based grading and student
evaluations, there are circumstances where competency-based education inspires
terrrific learning experiences.
Bob Jensen's threads on competency-based assessment ---
http://faculty.trinity.edu/rjensen/Assess.htm#ConceptKnowledge
Center for Research on Learning and Teaching ---
http://www.engin.umich.edu/teaching/crltengin/researchscholarship/index.html
"More Faculty Members Adopt 'Student Centered' Teaching," Chronicle
of Higher Education, October 18, 2009 ---
http://chronicle.com/article/Chart-More-Faculty-Members/48848/
Professors are
warming to new methods of teaching and testing
that experts say are more likely to engage
students, a UCLA survey found last year. Below
are percentages of faculty members who said they
used these approaches in all or most of the
courses they taught. Those trends may continue,
UCLA says, as full professors retire. Assistant
professors were much more likely, for example,
to structure teaching around small groups of
students, while full professors were more likely
to lecture extensively.
| |
2005 |
2008 |
|
Selected
teaching methods |
|
Cooperative learning (small groups of
students) |
48% |
59% |
|
Using
real-life problems* |
n/a |
56% |
|
Group
projects |
33% |
36% |
|
Multiple
drafts of written work |
25% |
25% |
|
Student
evaluations of one another’s work |
16% |
24% |
|
Reflective writing/journaling |
18% |
22% |
|
Electronic quizzes with immediate
feedback in class* |
n/a |
7% |
|
Extensive
lecturing (not student-centered) |
55% |
46% |
|
Selected
examination methods |
|
Short-answer exams |
37% |
46% |
|
Term and
research papers |
35% |
44% |
|
Multiple-choice exams |
32% |
33% |
|
Grading
on a curve |
19% |
17% |
|
* Not
asked in the 2005 survey |
|
Note:
The figures are based on survey
responses of 22,562 faculty members
at 372 four-year colleges and
universities nationwide. The survey
was conducted in the fall and winter
of 2007-8 and covered full-time
faculty members who spent at least
part of their time teaching
undergraduates. The figures were
statistically adjusted to represent
the total population of full-time
faculty members at four-year
institutions. Percentages are
rounded. |
|
Source: "The American College
Teacher: National Norms for the
2007-8 HERI Faculty Survey,"
University of California at Los
Angeles Higher Education Research
Institute |
Downfall of Lecturing ---
http://faculty.trinity.edu/rjensen/assess.htm#DownfallOfLecturing
Bob Jensen's threads on higher education are at
http://faculty.trinity.edu/rjensen/HigherEdControversies.htm
The
problem is that our students choose very bland, low nourishment diets in our
modern day smorgasbord curricula. Their concern is with their grade averages
rather than their education. And why not? Grades for students and turf for
faculty have become the keys to the kingdom!
Bob Jensen
"Are Undergraduates Actually Learning Anything?" by Richard Arum and
Josipa Roksa. Chronicle of Higher Education, January 18, 2011 ---
http://chronicle.com/article/Are-Undergraduates-Actually/125979/
Drawing on survey responses, transcript data, and
results from the Collegiate Learning Assessment (a standardized test taken
by students in their first semester and at the end of their second year),
Richard Arum and Josipa Roksa concluded that a significant percentage of
undergraduates are failing to develop the broad-based skills and knowledge
they should be expected to master. Here is an excerpt from Academically
Adrift: Limited Learning on College Campuses (University of Chicago Press),
their new book based on those findings.
Continued in article
Our Compassless Colleges: What are students really not learning?
http://faculty.trinity.edu/rjensen/HigherEdControversies.htm#Berkowitz
"The Case Against Case Studies: How Columbia's B-school is teaching
MBAs to make decisions based on incomplete data," by Geoff Gloeckler,
Business Week, January 24, 2008 ---
http://www.businessweek.com/magazine/content/08_05/b4069066093267.htm?link_position=link1
Shortly after R. Glenn Hubbard took over as
dean of Columbia Business School in 2004, he began hearing rumblings
from executives about the quality of MBA graduates. They were
undoubtedly smart but often unprepared to handle the most crucial of
managerial responsibilities: quickly solving problems with less than
perfect information. Among those wanting more from new hires is Henry
Kravis, co-founder of the private equity firm Kohlberg Kravis Roberts.
"I want to see MBAs who can jump in and make decisions, not jump in and
learn to make decisions," he says.
Hubbard made his own executive decision. He
devised a new twist on the case study—the teaching format invented by
Harvard Business School almost a century ago and used by most B-schools.
Hubbard's so-called decision brief offers less information about a
situation than the case study, and it doesn't present the solution until
students have grappled with the issues on their own. "We want our
students to be used to dealing with incomplete data," Hubbard says.
"They should be able to make decisions out of uncertainty."
Even Michael J. Roberts, the executive director
of the Arthur Rock Center for Entrepreneurship at Harvard and author of
more than 100 HBS case studies, acknowledges the potential benefits of
Hubbard's approach, which was introduced to Columbia students last fall.
"Framing problems and finding the data to analyze those problems is a
skill that MBAs need and that the classic case doesn't fully exploit,"
Roberts says. Hubbard expects such endorsements, as well as those of
companies, will encourage other business schools to make room one day
for Columbia's decision briefs in their curriculums. Hubbard, at least
initially, doesn't plan to sell the decision briefs but to use them to
tap into faculty research.
Hubbard isn't giving up on the traditional case
study altogether. As part of an initiative called CaseWorks, Columbia
will produce cases designed to reflect contemporary issues (which other
schools do already), while also creating decision briefs that do away
with the Harvard formula (which no one else has done). To help guide the
program, Hubbard has turned to two people familiar with the deficits of
the old methods: Stephen P. Zeldes, who has been at Columbia for more
than a decade and is now chairman of the economics department at the
B-school, and former Harvard case writer Elizabeth Gordon.
TOO MUCH INFORMATION The stock case study
presents a tidy narrative arc, with a protagonist and a clear story
line. One of the more widely used HBS cases focuses on Intel's (INTC)
former marketing vice-president, Pamela Pollace, as she decides whether
Intel should extend the "Intel Inside" branding campaign to products
other than computers. In 24 pages, students are provided with
information on Intel and the history of microprocessors, as well as
details about market share and segmentation. Pollace's major concern,
they learn, is brand dilution; the potential reward is likely worth the
risk. In effect, the students are guided along the decision-making
process.
If this case were a Columbia decision brief,
students might see a video interview in which Pollace describes the
challenge. They would also be given a few documents on the background of
the campaign itself—the same data a manager at the company would have,
but no more. Then, students would discuss possible solutions. Afterward,
the group would see a second video of Pollace explaining how she handled
the issue before debating whether or not she made the right decision.
So far, Columbia has produced six briefs that
take on of-the-moment business challenges: Among them is one that
focuses on General Electric's (GE) business-process-outsourcing division
in India. Given increased competition, the company needed to consider a
bigger investment, as well as the possibility of serving non-GE
customers. With just a little more information than that, students are
asked to come up with various strategies. "The idea is to try to
simulate what it will be like in a real workplace," says Gordon. "There
is uncertainty, things aren't predigested, all the information won't be
there."
The first field test for the new teaching
technique will be this summer, when the MBAs head out to their
internships. At Goldman Sachs (GS), which hires more Columbia interns
than any other company, the co-head of campus recruiting, Janet Raiffa,
hopes to encounter students who are more independent thinkers. As for
Kravis, his firm doesn't employ summer interns.
In some ways the pedagogy proposed by Columbia is a shorter and
cheaper version of the BAM approach first proposed by Catanach, Croll, and
Grinacker. The BAM approach uses a year-long case and students can seek out
data in virtually every way they will do it later on while on the job
(including paying for data if necessary) --- See Below
Bob Jensen's threads on Learning at Research Schools Versus "Teaching
Schools" Versus "Happiness" With a Side Track into Substance Abuse ---
http://faculty.trinity.edu/rjensen/Theory01.htm#Happiness
Learning Styles Sites
January 1, 2009 message from Pat Wyman
[raisingsmarterchildren@gmail.com]
Hello Bob,
Happy New Year! Your name came up through a google
alert, attached to my website and the complimentary learning styles
inventory at
http://www.howtolearn.com
It is on your page, from the community at
http://www.elearninglearning.com/learning-styles/microsoft/&query=www.howtolearn.com
I want to thank you for this is and if there is any way I can contribute
to your blog and yours to mine, articles, interviews, etc. I'd love to
connect with you.
You're doing wonderful work!
Warmly,
Pat Wyman, M.A.
-- Pat Wyman Best selling author, Learning vs.
Testing Co-Author,
Book Of The Year In the Medicine Category, The Official Autism 101 Manual
University Instructor of Continuing Education, California State University,
East Bay Founder,
http://www.HowToLearn.com and
http://wwwRaisingSmarterChildren.com
Winner, James Patterson PageTurner Award Get your copy of Learning vs.
Testing with complimentary materials at http://www.learningvstesting4.html
Get Tips For Raising A Smarter Child at
http://www.RaisingSmarterChildren.com
"There are two ways you can live your life - one as
if nothing is a miracle, and the other as if everything is a miracle."
Albert Einstein
Bob Jensen's threads on assessment and learning ---
http://faculty.trinity.edu/rjensen/assess.htm
Bob Jensen's threads on metacognitive learning ---
http://faculty.trinity.edu/rjensen/265wp.htm
Bob Jensen's threads on asynchronous learning ---
http://faculty.trinity.edu/rjensen/255wp.htm
Bob Jensen's threads on education technology ---
http://faculty.trinity.edu/rjensen/000aaa/0000start.htm
February 17, 2005 message from Bob Jensen
I call your attention to Page 4 of the Spring 2005 newsletter called “The
Accounting Educator” from the Teaching and Curriculum Section of the
American Accounting Association --- http://aaahq.org/TeachCurr/newsletters/index.htm
The current Chair (Tomas Calderon) has a piece about “reflection”
which is nice to reflect upon. There are abstracts of papers in other
journals that relate to education, and an assortment of teaching cases.
Marinus Bouman has a nice piece entitled “Using
Technology To Integrate Accounting Into The Business Curriculum.”
Interestingly, the Sam M. Walton College of Business at the University of
Arkansas no longer has courses in Principles of Accounting (or Marketing or
Finance). You should read Bouman’s article to find out what took the place
of these principles courses in a daring curriculum experiment.
Since I teach accounting theory, I found Bob Clusky’s paper “Where’s
Accounting Theory) quite interesting. Even more than AIS, “Accounting
Theory” is a phrase still in search of a definition.
Tim Fogerty has a piece on Distance Education. It is somewhat negative in
tone, but Tim seems to sigh that the march forward is inevitable and the
current boundries of education from one program or one institution will
evaporate as students seek courses and modules from anywhere in the world. I
might take issue with some of his conclusions such as testing and/or
assessment, but this is not the time or the place for that. See http://faculty.trinity.edu/rjensen/assess.htm
There is much, much more of interest in this 32 page newsletter. Go to http://aaahq.org/TeachCurr/newsletters/index.htm
Page 4 describes a forum to be headed up by Tim Fogarty (Case Western)
and Call for Papers in which the last paragraph reads as follows:
*******************************
Issues in Accounting Education, in conjunction with the Teaching &
Curriculum Section of the AAA has asked me to edit a dedicated forum with
an expected publication date of Spring 2006. I would like to extend an
opportunity to accounting educators to submit essays for this issue.
Proposed pieces for inclusion should be 25 pages (double spaced) or less.
Submissions will be peer-reviewed with an emphasis on clarity and strength
of ideas. The deadline for the first drafts is March 1, 2005. There would
also be an opportunity to discuss these ideas in a CPE session at the AAA
meeting in San Francisco.
*****************************
Bravo to Thomas, Tim, and other volunteers who are continuing the
momentum of this essential section of the AAA! This is the lifeblood of why
we are in this profession.
Bob Jensen
February 17, 2005 message from Bob Jensen
Note
the following paragraph that I wrote in my previous message:
*********************
Marinus Bouman has a
nice piece entitled “Using Technology To Integrate
Accounting Into The Business Curriculum.”
Interestingly, the
Sam
M.
Walton
College
of Business at the
University
of
Arkansas
no longer has courses in Principles of Accounting (or Marketing or Finance).
You should read Bouman’s article to find
out what took the place of these principles courses in a daring curriculum
experiment.
*******************
The simulation pedagogy used in the "Business Foundations"
course at the Walton College seems to be quite related to the BAM pedagogy ---
http://faculty.trinity.edu/rjensen/265wp.htm#UVA000
The
more I think about this, the more I think that the
Walton
College
as evolved, I assume quite independently, into something quite similar to what
Jack Wilson (a physicist) pioneered at
Rensselaer
well over a decade ago. Core
courses (such as physics) in the general curriculum at
Rensselaer
were taken from the curriculum and replaced with technology-based “studio”
learning. The
University
of
Arkansas
is doing something similar in relying upon technology when taking the core
courses, such as Principles of Accounting, out of business education.
The
following is taken from http://www.trinity.edu/~rjensen/245glosf.htm#Studio1
Studio
classroom= An application of computer technology
pioneered by Jack M. Wilson at Rensselaer Polytechnical
Institute for replacing large lecture courses with students working in pairs
in front of computer screens where they interactively tackle problems and
issues rather than listen to or passively watch lectures in front of a mass
lecture section. The only lecture comes at the beginning and end of class
where the instructor commences or wraps up the learning session. The
"studio" is a combination lab and electronic classroom.
Dr.
Wilson now serves as the President of the
University
of
Massachusetts
system. He had been serving as the Vice President for Academic Affairs
of the
University
of
Massachusetts System
and is the founding Chief Executive Officer (CEO) of UMassOnline,
the University of
Massachusetts Virtual University. As Vice president he was
responsible for the coordination of the academic programs in research and
teaching throughout the five campus system. As CEO of UMassOnline
he worked with the five physical campuses,
Amherst
,
Lowell
,
Boston
,
Worcester
, and
Dartmouth
to provide online access to the programs of the
University
of
Massachusetts
.
Jack
Wilson was one of the early pioneers in education technologies and learning.
He is now CEO and founder of UMass Online
.
Dr.
Wilson, also known as an entrepreneur, was the Founder (along with Degerhan
Usluel and Mark Bernstein), first President, and
only Chairman of LearnLinc
Corporation (now Mentergy),
a supplier of software systems for corporate training to Fortune 1000
Corporations. In early 2000. LearnLinc
merged with Gilat
Communications, (GICOF)
which also acquired Allen Communication
from the Times Mirror group. The Gilat-Allen-LearnLinc
combination forms a powerful "one stop shopping" resource for
E*Learning that is now the Mentergy unit of Gilat
Communications. (The LearnLinc
Story).
Dr
Wilson was the J. Erik Jonsson '22 Distinguished
Professor of Physics, Engineering Science, Information Technology, and
Management and the Co-director of the
Severino
Center
for Technological
Entrepreneurship at
Rensselaer
. After coming to
Rensselaer
in 1990, he served as the
·
Dean
of Undergraduate Education,
·
Dean
of Professional and Continuing Education,
·
Interim
Provost,
·
Interim
Dean of Faculty, and as the
·
Founding
Director of the
Anderson
Center
for Innovation in
Undergraduate Education.
In
these roles, Wilson led a campus wide process of interactive learning and
restructuring of the educational program, known for the design of the Studio
Classrooms, the growth of the Distributed Learning Program, the creation of
the Faculty of Information Technology, and the initiation of the student
mobile computing (universal networked laptop) initiative
The
Studio Classrooms at
Rensselaer
replaced large sized core courses taught by traditional lecture pedagogy with
student teams responsible largely for teaching themselves using computer-aided
and interactive course materials --- http://www.rpi.edu/dept/NewsComm/WNCTW/ad7.html
Welcome
To Interactive Learning
Roll up your sleeves and take a seat in the
Rensselaer
studio classroom. Classes
of about 60 students are engaged at wired workstations - utilizing cutting
edge tools like Web-based technologies, full-motion video, computer
simulation, and other laboratory resources. An
instructor and teaching assistant move from workstation to workstation
observing and coaching. Notes are taken with a simple mouse click, as
students download files and class materials onto their required laptops.
It's an innovative blend of discussion and skill-building, high-tech inquiry
and problem-solving - preparing scholars to succeed in the new business
world. It's all part of Interactive Learning at
Rensselaer
.
More
Studios Than
Hollywood
Interactive Learning is more than just a concept at
Rensselaer
; it's a working reality.
The approach has been infused throughout all of our undergraduate
disciplines in more than 25 studio classrooms with more being built all the
time. In the LITEC studio classroom, students build remote-controlled cars
in a project-based, team environment. In the Circuits Studio, students
develop and test their own circuits. The Collaborative
Classroom, funded by the National Science Foundation, serves as a testbed
for using computer technology to collaborate on design projects. At
Rensselaer
, knowledge and application
are seamlessly intertwined.
Teaching
How We Teach
Rensselaer
's revolutionary model for
education has been talked about, honored, and emulated. We earned the first
Pew Charitable Trust Award for the Renewal of Undergraduate Education and
the first Boeing Outstanding Educator Award, among others. Last year, we
were named to administer an $8.8 million Pew-funded program to bring
educational innovation to other universities in this country: The
Center for Academic Transformation. Literally hundreds of institutions
have visited
Rensselaer
to learn how we teach.
No
Stopping Now
Of course, the very thinking that enabled
Rensselaer
to initiate Interactive
Learning is the same mindset that keeps us pressing forward.
Rensselaer
's Anderson
Center for Innovation in Undergraduate Education was founded 11 years
ago with the continuing mission of making
Rensselaer
a leader in innovative
pedagogy. More recently, the
Rensselaer
Academy
of Electronic Media has become the spawning ground for highly creative
visualization software that enables students to learn scientific and
engineering principles in ways never before possible. We continue to look
for new and better methods to evolve education - meeting the present and
future needs of our students, professors, and global businesses. Because
solving real-world challenges is our mission and
our passion.
For a
summary short summary see http://www.bc.edu/bc_org/avp/soe/cihe/newsletter/News15/text4.html
. (See also Electronic
classroom)
Random Thoughts (about learning from a retired professor of
engineering) ---
http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Columns.html
Dr. Felder's column in Chemical Engineering Education
Focus is heavily upon active learning and group learning.
Bob Jensen's threads on learning are in the following links:
http://faculty.trinity.edu/rjensen/assess.htm
http://faculty.trinity.edu/rjensen/255wp.htm
http://faculty.trinity.edu/rjensen/265wp.htm
Metacognitive Concerns
in Designs and Evaluations
of Computer Aided Education and Training: Are We
Misleading Ourselves About Measures of Success?
I've gotten to the point where I really don't lecture
anymore, so if I look awkward up here talking to you, it's because I haven't lectured in
four years. And the second point is that I've lost total faith in lecturing. Try telling
your students something important in class, and then finding out how many heard it to
where they can remember it the next day.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
Caveat: . I am grateful to
Professors Croll and Catanach for allowing me to videotape their inspiring presentation.
The quotations from Professors Croll and Catanach that appear at various points in this
document have never been edited by those professors or modified from a transcript of a
presentation that I videotaped at a conference. My videotape was transcribed by my
secretary, Debbie Bowling. The transcription
was modified by me only when Debbie failed to understand certain terminology. I
prefer to minimize changes in the transcription so that what is read remains as close as
possible to what the audience listened to at the conference. None of us speak with the
formalized vocabulary and grammar used in our writing. Also we cannot edit what we said in
the same manner that we can edit what we wrote. Hence, transcriptions should not be judged
as writing. In August 1998, Tony Catanach moved to Villanova University in
Villanova, Pennsylvania.
Introduction
R. Jensen (1998a) documents
the exponential increase in asynchronous learning where students take on greater
responsibility for teaching themselves and set their own learning paces. Evidence is
mounting that student performance thereby improves. Some of our finest and oldest
universities in the world are experimentally replacing lectures with asynchronous learning
networks where students no longer attend scheduled classes. The early success of the Sloan
Foundation funded experiments, where traditional lectures are replaced by asynchronous
learning networks (ALNs), are making all institutions take notice even for full-time
resident students. Examples of these successes and concerns about these successes are
documented in R. Jensen (1998a).
In addition, traditional (synchronous) lectures and cases are taking place in electronic
classrooms where live Internet connections and presentation multimedia are replacing chalk
and flip charts. Students in electronic classrooms or in front of their own laptops at
home can be taken into virtual learning worlds and electronic chat rooms that make
learning easier, faster, more collaborative, and more fun. Virtual worlds make learning
more contextual. Biology students work with virtual organs and organisms, chemistry and
physic students enter virtual laboratories, medical students diagnose and heal virtual
patients, sociology students colonize virtual societies, finance students choose
portfolios in virtual markets, etc.
Across many years and nations, educators have long known that interactive learning
tends ceteris paribus to be better than passive learning. Computer and networking
technologies make interactive learning more effective and efficient. Students having
keyboard, mouse, joystick, microphone, web camera, and other input controls have greater
powers of interaction with instructors, other students, libraries, experts, and objects
around the world. Students not only can listen to Beethoven and Chopin, they can be
transported back in time to the virtual lives and times of master composers. The great
libraries of online documents, major university library holdings, and government archives
are at their fingertips.
Jensen and Sandlin (1998, Chapter 2) list many advantages of
newer learning technologies. R. Jensen (1998a) outlines various
worries and concerns. The main purposes of this paper are as follows:
- to point out several "metalevel" concerns about newer learning technologies;
- to discuss learning material design considerations for newer technologies that may
improve long-term memory and performance of students. In particular, we face the challenge
of designing greater ambiguities and learning
frustrations into computer-aided learning materials that tend to reduce rather than
increase ambiguities into learning tasks.
- To caution about metacognitive risks (possibly paradoxes) in evaluating programs,
courses, and learning modules prior to graduation and even earliest years of post-graduate
work and study.
"The Great Debate: Effectiveness of Technology in Education," by
Patricia Deubel, T.H.E. Journal, November 2007 ---
http://www.thejournal.com/articles/21544
According to Robert Kuhn (2000), an expert in brain
research, few people understand the complexity of that change. Technology is
creating new thinking that is "at once creative and innovative, volatile and
turbulent" and "nothing less than a shift in worldview." The change in
mental process has been brought about because "(1) information is freely
available, and therefore interdisciplinary ideas and cross-cultural
communication are widely accessible; (2) time is compressed, and therefore
reflection is condensed and decision-making is compacted; (3) individuals
are empowered, and therefore private choice and reach are strengthened and
one person can have the presence of an institution" (sec: Concluding
Remarks).
If we consider thinking as both individual
(internal) and social (external), as Rupert Wegerif (2000) suggests, then "[t]echnology,
in various forms from language to the internet, carries the external form of
thinking. Technology therefore has a role to play through supporting
improved social thinking (e.g. providing systems to mediate decision making
and collective reasoning) and also through providing tools to help
individuals externalize their thinking and so to shape their own social
worlds" (p. 15).
The new tools for communication that have become
part of the 21st century no doubt contribute to thinking. Thus, in a debate
on effectiveness or on implementation of a particular tool, we must also
consider the potential for creativity, innovation, volatility, and
turbulence that Kuhn (2000) indicates.
Continued in article
Bob Jensen's threads on education technology are at
http://faculty.trinity.edu/rjensen/000aaa/0000start.htm
Bob Jensen's threads on assessment are at
http://faculty.trinity.edu/rjensen/assess.htm
Metamemory and Metacognition: The Metalevel
Activities of the Brain
The brain stores long-term memory in the outer cerebral cortex with complex
recollection systems among memory patterns of billions of neurons. Most things stored
therein cannot be instantly recalled, but ways in which knowledge becomes embedded and
patterns of use over time affect if and when recall takes place. Much depends upon
associations between stored knowledge. For example, I probably would never recall the name
of Don Quixote’s horse had this not been a memorable question (for me) on a 1961 GRE
examination. That trivia question "sticks" in my mind all these years because of
the humor of being asked about the only thing I remembered from that book (last read while
I was a high school farm boy who liked horses).
There are also important distinctions between visual versus verbal memory, and these
distinctions are crucial in designs of asynchronous learning materials. In fact, virtually
all of Howard Gardner's theories on seven types of intelligence are important to take into
account in the design of learning materials. These are described as follows by The Gardner
School at http://www.swopnet.com/ed/TAG/7_Intelligences.html
:
1. Linguistic
Children with this kind of intelligence enjoy writing, reading, telling stories or
doing crossword puzzles.
2. Logical-Mathematical
Children with lots of logical intelligence are interested in patterns, categories and
relationships. They are drawn to arithmetic problems, strategy games and experiments.
3. Bodily-kinesthetic
These kids process knowledge through bodily sensations. They are often athletic,
dancers or good at crafts such as sewing or woodworking.
4. Spatial
These children think in images and pictures. They may be fascinated with mazes or
jigsaw puzzles, or spend free time drawing, building with Legos or daydreaming.
5. Musical
Musical children are always singing or drumming to themselves. They are usually quite
aware of sounds others may miss. These kids are often discriminating listeners.
6. Interpersonal
Children who are leaders among their peers, who are good at communicating and who seem
to understand others’ feelings and motives possess interpersonal intelligence.
7. Intrapersonal
These children may be shy. They are very aware of their own feelings and are
self-motivated.
However, designing multimedia with the above types of intelligence in mind does not in
and of itself aid to metacognition and metamemory. The mysterious process of
"deciding" (it’s an unconscious process) what gets stored versus what gets
lost or discarded takes place in a deeply embedded part of the brain known as the
hippocampus. That organ is analogous to a computer’s file manager when it stimulates
neurons to form an episodic network of neurons. Two important criteria affecting this
process are emotional significance and association significance (associations with
episodes already embedded in neuron structures). Information overload of the brain is not
so much an issue of the number of neurons as it is with the input rate and complexity of
processing incoming perception vis-à-vis emotions and episodic associations. Short-term
memory "junk" such as a seldom-used phone number never clutters up our long-term
memory episodic networks. Rote memorization of the phone number may last for a day but is
not likely to last for a decade or even a week after it is used. When the hippocampus is
damaged, a person might never store incoming knowledge and live only with recordings made
when the hippocampus functioned properly. Alzheimer’s disease attacks the hippocampus
region of the brain. Aaron, Benjamin, and Bjork (1998, p. 55) define metamemory as follows:
There has been a surge of interest in metamemory --- the study of
what people know and understand about their own memory and memorial processes. From a
theoretical standpoint, there has been a particular effort to explain why certain
metamnemonic measures, such as the feelings of knowing . . . are accurate or
inaccurate under various conditions.
The term "cognition" refers to the entire process of knowing --- a process
comprised of perception, concentration, memory, judgment, and awareness. A subset of
cognition known as metacognition is set apart from most cognitive activities and takes
place in the frontal lobes. Metacognition is little understood even though we do know that
it is impacted by episodic association formations in the hippocampus and that our
well-developed metacognition is what sets human beings apart from other species.
Psychologists refer metacognition as the monitoring process superimposed upon the control
process of cognition. This monitoring can alter the control processes. Metacognition gives
rise to what is commonly referred to as "Feeling of Knowing" or FOK judgments in
cognition literature. It also gives rise to what is known as "novelty" versus
"familiarity" awareness of perceptions.
Thiamine deficiency in the frontal lobes often accompanies severe alcohol abuse and is
known in medicine as the Korsakoff Syndrome. Korsakoff amnesia thereby differs from
amnesia that does not so dramatically affect the frontal lobes. Shimamura
and Squire (1986) report that patients diagnosed with Korsakoff Syndrome have impaired
FOK judgments. The theory is that frontal lobes processes modulate the hippocampus
processes of forming episodic associations. Cognitive scientists make a distinction
between basic memory and metamemory. Metamemory helps to make distinctions between stored
memories. Weingradt, Leonesio, and Loftus (1994) note that a witness to an
automobile accident may store perceptions at the scene of the accident and knowledge from
what he or she read about or heard subsequent to the event. On Page 183 they elaborate as
follows:
Consider a typical situation in which an eyewitness is asked to report
what he or she has seen. Whether the witness is asked to identify a perpetrator, describe
the events leading up to an automobile accident, or discriminate between information
actually witnessed in an event or read about in a newspaper article, he or she is required
to make judgments about information in memory. In metacognitive terms then, eyewitnesses
are often required to make metamemory monitoring judgments. As should be evident from our
discussion of the misinformation effect as a metacognitive error, the literature on
metamemory monitoring can provide valuable insights into the reasons for poor eyewitness
memory performance.
These metalevel processes enable humans to reflect upon thoughts and behaviors. Nelson and Narens
(1994) refer to "metalevel" processes that are set apart from the
lower-order "object-level" memory and cognition processes. Metalevel processes
monitor perceptions to assess their novelty/familiarity and to direct attentiveness and
feelings. Janet Metcalfe (1996,
pp. 381-382) quotes a passage from Plato in which Socrates refers to an
"inner voice." She then goes on to write the following passage:
Socrates' inner voice is the earliest well-known example of a
self-reflective consciousness --- an internal monitor and control system. A
self-monitoring consciousness figures in the musings of Descartes, emerging as "the
thinker" whose existence cannot be doubted, even though all the content of the
thought is open to question."
Later on Page 382 she continues as follows:
An accumulating body of evidence points to the prefrontal cortex as a
brain region of critical importance for these [metalevel] functions. This region is
distinct from those areas responsible for the knowledge and operations upon that knowledge
that are the object of reflection, though, of course, they are interconnected. As Nelson and Narens
(1994) have pointed out, object-level functions of cognition, such as memory storage,
retrieval, and applying the operations of problem solving, may be separable from metalevel
functions that monitor and control the object-level functions. In the normal person these
two levels, cognition and metacognition, interact in a complex manner. This interaction is
critical not only for problem solving, planning, and for memory, but arguably for
sensitive assessments of one’s own appropriate social behavior.
Metacognition is essential to what earlier writers have called introspection. Nelson and Lorens
(1994, p. 17) quote William James in 1890 as follows:
Introspective Observation is what we have to rely on first and
foremost and always. . . . I regard this belief as the most fundamental of all postulates
of Psychology.
Object-level versus metalevel distinctions become important to educators for
self-evident reasons. The implications are immense. For example, it may suffice to make it
as easy as possible (e.g., with visual aids) for students to memorize multiplication
tables, but we are not so certain that it is best to make it as easy as possible to
understand number theory theorems. Quite possibly students lose introspective powers if
they do not re-discover some aspects of number theory on their own rather than start at
the top of the heap of known theory. Some aids to making it easier to understand the
theorems may be detrimental to long-term memory and performance.
Making Learning More Easy, Fun
and Collaborative: Are We
Taking Things Too Far in Virtual Learning Worlds?
I have been making hypermedia CD-ROMs for my students since 1992. Until recently, my
goals have been to make learning more easy, fun, inspirational, realistic, collaborative,
and efficient. I take my students into worlds of virtual financial contracting and
reporting. When seeking answers, my students can watch and listen to world experts explain
complex transactions (e.g., financial instruments derivatives contracts and mathematical
models for analyzing investment risks) and issues and theories about accounting for those
transactions. I have videotaped experts at numerous conferences and workshops. Then I
digitized the experts’ audio/video excerpts onto CD-ROMs and organized the material
for pedagogic ease and efficiency. I also transcribed hundreds of hours of audio into text
that can be searched using key words and is presented in a variety of formats (including
Jeopardy-type games that make learning more fun). Readers can view some of my course
materials from links at my web site.
I also made things easier by obtaining rights to share commercial literature databases
and CPA Examination review courses on CD-ROMs. For example, by using the Price-Waterhouse
Researcher CD-ROM, the accounting and auditing standards of the major developed nations
are literally at the tips of student fingers on a keyboard. I can make my assignments
exceedingly tough and realistic. Students can find the answers using my CD-ROMs, network
server files, the Internet, textbooks, and photocopy handouts. I created a learning world
in which my students can quickly find answers and do not have to conduct those slow,
frustrating, and serendipitous library searches. When computer coding became involved for
students (e.g., to create Microsoft Access databases or to code JavaScript web documents),
I wrote tutorials that made learning much easier.
My goal was to take students to heights that were not possible in my courses before
computing and networking hypermedia technologies. By making the hunt for answers easier
and fun and collaborative, I could cover more material and more difficult material. My
courses were not necessarily easier since I added more material. However, my courses were
more efficient since students learned faster and spend less time searching for answers and
completing projects. Learning became easier and in some respects more fun since student
could see and hear experts whose names appear in the literature.
My own research, however, is far more serendipitous than research than I require from
my students. Once in a while I pull myself away from the Internet and wander amongst the
shelved books in the library. I peruse tables of contents and take a glance at sections of
books and journal contents that capture my eye (I like to think of it as metacognitive
success in identifying something novel that is not familiar to me). An article that I
actually stumbled upon in the manner described above has had a profound impact upon my
teaching and research. That article is "Memory and Metamemory Considerations in
the Training of Human Beings" by Bjork (1994). Dr. Bjork convinced me that I was setting some wrong
goals when creating asynchronous learning materials for my courses. In addition, he
convinced me that many of the things that I had written about education technology are
wrong.
I do not go so far as to say that the goals of using new technologies for making
learning more easy, fun, inspirational, collaborative, and efficient are wrong per se.
I do, however, suspect that there are dangers in making these goals too sacrosanct in
preparing learning materials for lectures and asynchronous learning outside the classroom.
By way of illustration, let me focus on a passage from Bjork (1994, p. 197-198).
The process of accessing stored information given
certain cues also does not correspond to the "playback" of a typical recording
device. The retrieval of stored information is a fallible, probabilistic process that is
more inferential and reconstructive than literal. . . . The information in our long-term
memories that is, and is not, accessible at a given point in time is heavily dependent on
the cues available to us, not only on cues that explicitly guide the search for the
information in question, but also on environmental interpersonal, mood-state, and
body-state cues.
In years past, I created cases on the computer that outlined financial contracting
situations. For example, a typical case would entail an interest rate and/or foreign
currency risk exposure that a company hedged with circus swaps (combinations of interest
rate and currency swap financial instruments derivatives contracts). In this virtual world
of contracting, I then gave my students links to my own learning materials and Internet
links that provided the information needed to assess and account for risks. They had to
find the solutions and then be prepared to explain those solutions.
After pondering Bjork (1994)
and related references on metacognition, I decided that my students were too dependent
upon virtual worlds (cases) that I created and solutions that I provided in the course
materials. Now I make my students write their own cases and propose their own solutions.
Extra credit is given for attacking issues for with there are no known solutions. Credit
is given to finding sources that I do not provide and, in many instances, have never
encountered. Extra credit is given for searches outside the fields of accounting and
finance. My goal is to multiply to encode the learning and retrieval process. Bjork (1994, p. 189) writes the
following:
On the encoding side, we would like the learner to achieve, for
lack of a better word, an understanding of the knowledge in question, defined as an
encoding that is part of a broader framework of interrelated concepts and ideas. Critical
information needs to be multiply encoded, not bound to single sets of semantic or
situational cues. . . . Similar to the argument for multiple encoding, it is also
desirable to induce successful access to knowledge and procedures in a variety of
situations that differ in cues they do and do not provide.
Whether learning contexts are defined as "cases" or "virtual
simulations" or "virtual games," the important point to consider in their
designs is that they may be too context specific and/or lack variety to the detriment of
long-term memory and performance. Our students may spend too much time mastering solutions
that we make available to them albeit in some form of treasure hunts. Our cues may be
too singular and too situational in the computer worlds that we present to our students.
Metacognition ---
https://en.wikipedia.org/wiki/Metacognition
Reading Apprenticeship at WestEd: Downloadable Resources (metacognition) ---
http://readingapprenticeship.org/publications/downloadable-resources
Metacognitive Bookmark ---
http://readingapprenticeship.org/wp-content/uploads/2014/01/RFU-ch-4-metacog-bookmark.pdf
Stanford Researchers Discover a Smarter Way to Prepare for Exams: Introducing
MetaCognition, the Art of Thinking About Your Thinking ---
http://www.openculture.com/2017/05/metacognition-the-art-of-thinking-about-your-thinking.html?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+OpenCulture+%28Open+Culture%29
The BAM Pedagogy at the University of
Virginia: Is BAM's
Success Due to Metacognition and Metamemory Phenomena?
Year 2004 Update
Congratulations
to Tony Catanach and Noel Barsky from Villanova University
Tony and Noel were
awarded the 2004 American Accounting Association Innovation in Accounting
Education Award in Orlando on August 11, 2004. The award is for their
development of a simulation model for teaching management accounting. The
model is called the Business Planning Model and is available with the textbook Management
Accounting , A Business Planning Approach (Houghton Mifflin) --- http://snipurl.com/CatanachBPM
Designed for use in introductory or graduate-level
managerial accounting courses, this text applies an objective-based approach
to managerial accounting topics. Unlike traditional cost-accounting texts, Management
Accounting emphasizes the critical role that information plays in decision
making, strategy execution, and overall enhancement of a firm's value. This
text meets the growing demand for an integrated, "survey of
business" approach to managerial accounting.
Through problem-based learning and the business
planning model (BPM), Management Accounting develops in students those
competencies expected of today's business professionals. This innovative
pedagogical approach stresses the understanding and application of the basic
business process; risk assessment and its relation to business strategy;
critical thinking, reasoning, and analysis; oral and written communication
skills; and techniques for team building. Real-world business problems and
simulations place students in the role of business consultant.
- Each chapter includes mini-cases, exercises, and
problems built around the challenges faced by managers today. An
integrated text-long continuing example of a fictitious service business, C&F
Enterprises, simulates the actual strategic planning process, as well
as resource management and performance measurement activities commonly
found in business practice.
- Business Beacons boxed features provide a
bridge from chapter concepts to contemporary business examples by
highlighting actual companies and linking managerial accounting topics to
real-world examples.
- Net Gains sections at the end of each
chapter direct students to Internet resources that enrich chapter
material. Students can use after-class time to link to tools that perform
many common accounting computations, while spending more in-class time on
discussion and development of analytical skills.
- Each chapter concludes with a Business Planning
Application, an optional module that allows students to engage in a
semester-long business planning simulation.
- A detailed and flexible teaching guide offers
instructors additional resources, tips, and tools for implementing the
innovative Management Accounting approach.
- A comprehensive appendix--Preparing and
Presenting the Business Plan--integrates and expands upon the business
planning applications and concepts from the text. It provides students
with a detailed guide on how to prepare and present a business plan
through exercises and additional resources.
The authors
presented a CPE workshop on the BPM in Orlando --- http://aaahq.org/AM2004/cpe/cpe27.htm
The Business
Planning Model is an innovative extension of the Business Activity Model
approach to teaching Intermediate Accounting without lectures (the students must
learn on their own). I wrote a paper about the BAM model at http://faculty.trinity.edu/rjensen/265wp.htm#UVA000
The BAM pedagogy was developed at the University of Virginia (when Tony was on
the faculty at Virginia) as one of the Accounting Education Change Commission
funded projects.
On August 20, 1997, at the Annual Meetings of the American Accounting Association in
Dallas, I videotaped an extraordinary presentation by Professors David Croll and Anthony
Catanach from the McIntire School of Commerce at the University of Virginia. The topic was
the Business Activity Model (BAM) revolutionary and award-winning revision of the
Intermediate Accounting sequence in which all lectures and textbook assignments were
replaced by a two-semester focus on accounting for a business from its inception through
seven years of operation. Development of the BAM case was initially funded with $50,000
from the Accounting Education Change Commission (AECC). The AECC was set up to inspire
innovation and allocate funds given by the largest eight accounting firms to foster change
in accounting education pedagogy. In 1997, the BAM experiment won the prestigious $5,000
Innovation in Accounting Education Award administered by the American Accounting
Association. The main innovation of the BAM pedagogy is that
students teach themselves in a discovery learning pedagogy. The main purpose
of BAM is make students constantly confront ambiguity. They
are assigned to teams, but teams are not graded. Although spreadsheets and some other
learning materials are on line, this is not a high technology ALN application with chat
rooms, etc.
In my viewpoint this paper also has relevance to the findings of another AECC-funded
experiment called the Project Discovery (PD) project at the University of Illinois at
Urbana-Champaign. That study took a slightly different approach across multiple
accounting courses to study the effects of the importance of having students learn from
"complex, ill-structured, ambiguous problems and cases similar to those found in
practice" as reported by Stone and Shelly (1997, p. 26). Stone and Shelly repeatedly
stress that their research notes the impact of such factors on learning but make no
attempt to attribute causality. I speculate that some of the causal factors are
metacognitive. Even though my paper focuses on the BAM Program at the University of
Virginia, I think my conclusions extend to the PD Program at the University of Illinois.
The BAM pedagogy is a "success" in terms of a number of important criteria.
Students on internships and recent graduates in full-time jobs report that the BAM
learning context is much more like "what really happens" on the job. In
particular, they feel better prepared for dealing with ambiguities
of real life. Secondly, long-term memory seems to be enhanced by the BAM pedagogy. More
than a year passes between completion of Intermediate Accounting and sitting for the
national CPA examination. Performance on the CPA examination improved markedly using the
BAM pedagogy. Without having the traditional lecture/drill pedagogy students remember
better with the BAM learn-it-yourself pedagogy. Instructors using the BAM pedagogy claim
students do better. They also claim that weaker students that tend to have more trouble
with examinations are given more opportunities to show that they know more than is
measured on examinations. In the BAM pedagogy, students make presentations in front of
fellow team members and in front of the entire class. Students have more realistic and
memorable interactions in the BAM pedagogy. David Croll relates the following:
. . . afterwards she felt better about it, and
participated a lot more and seemed to be much stronger at it. In student groups we have
finance majors as well as accounting majors in this course, and some of the finance majors
are great at speaking out. And so I get one young lady who's going off to work for a
Big-Six CPA firm, and she had three finance majors, three real talkers, in her group. And
on this accounting issue, the three men were adamant that they had this correct answer and
she was wrong. It turns out that she really had the right answer --- and I felt great! She
stuck with it. She argued them down.
And it went on for about ten minutes. We stood back and let her
fight it out with them. She took them on --- all three. And she's gonna have to do that in
her professional career. Go out there and say, this sounds good. This is the correct
accounting. And afterwards, I think she had a lot more confidence and interest in going
out and doing her job.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
The BAM pedagogy across two semesters is revolutionary accounting education.
Intermediate Accounting in virtually all colleges is a textbook driven set of two or three
courses aimed at mastering rules for accounting for financial transactions. Since business
contracting and transactions have become so complex in the past three decades, the
complexity and volume of the rules and practices have burgeoned. Intermediate Accounting
classes are traditionally synchronous in terms of lectures and/or case discussions.
Learning under the BAM pedagogy is asynchronous both inside and outside the classroom. More importantly, the BAM approach makes students search for answers on
their own or in teams such that "they teach themselves."
I made a CD-ROM of the BAM presentation and feature the BAM pedagogy and outcomes
assessments in my education technology
workshops that I conduct on college campuses around the world. After
stumbling belatedly upon the Bjork
(1994) paper (subsequent to giving several workshops featuring the BAM pedagogy), it
dawned on me that reasons why faculty at the University of Virginia feel that the BAM
pedagogy is a resounding success relative to traditional pedagogy lie in metacognitive and
metamemory phenomena. In this paper I will try to make the case that some reasons for the
success of the BAM pedagogy lie in metalevel processes in the brains of students.
Please keep in mind that the success of the BAM experiment may be heavily due to the
skills and dedication of the faculty and the high quality of the students participating in
the experiments. It is not at all certain that other faculty can carry this off at other
places and times. It is not clear that these same faculty (Croll and Catanach) can
maintain the intensity and motivation year after year that were present in the first four
years of experimentation. The BAM pedagogy takes students and faculty into many issues for
which there are no known solutions and requires that faculty and students tolerate far
more ambiguity throughout the entire year of learning. Ambiguity creates stress for students and faculty.
According to David Croll, ambiguity is the reason Robert
Grinaker (original BAM case author who is now emeritus), David Croll, and Anthony Catanach
proposed making such a dramatic change in pedagogy for Intermediate Accounting. Graduates
prior to the BAM pedagogy graduated with an air of confidence that they had mastered
accounting rules and pat solutions for the real world. In the first year on the job they
discovered that these pat answers seldom applied in practice. They were not prepared for
the ambiguities and complexities of real world careers. Many of them changed careers or
returned to law schools according to Professor Croll. The problem is pervasive and is in
no way unique to accounting graduates from the University of Virginia. The accounting
firms are not happy because of the turnover in new employees, and the new employees are
not happy because they were not prepared to deal with unresolved problems and ambiguities
of their work assignments. Furthermore, newly minted graduates prior to the BAM pedagogy
were not trained and educated to deal with ambiguities and introspective talents needed
for complex business transactions. These are the major reasons the largest firms attempted
to foster accounting education change with AECC grants to colleges and universities.
Hence, one goal of the BAM pedagogy is to deal with more ambiguous issues prior
to graduation. These are invariably viewed as harder hurdles for students. Without knowing
it, the BAM developers were in fact conforming to what Robert Bjork claims is most
important for long-term performance. Bjork (1994, pp. 189-193) called this point "The Need to
Introduce Difficulties for the Learner." In particular, Bjork stresses the need to
introduce "variation and/or unpredictability," "contextual
interference," distributing the learning of topics over extended intervals of time,
reducing feedback, "using tests as learning events," and being willing take
risks of lowered evaluations of instructors.
UVA010
The Business Activity Model at the University of Virginia:
The Need to Introduce Difficulties for the Learner
Whatever the exact mixture of manipulations that might turn out
to be optimal, however, one general characteristic of that mixture seems clear: It would
introduce many more difficulties and challenges for the learner . . . . Recent surveys of
the relevant research literature . . . leave no doubt that many of the most effective
manipulations of training --- in terms of post-training retention and transfer --- share
the property that they introduce difficulties for the learner.
Bjork (1994, p. 189)
Metacognition: Knowing about knowing
Now one of the problems is Bob wrote the case
without thinking about the accounting. If I ever wrote a case, I'd write a case about what
I knew. He wrote a case about a business he knew. And so there were times when he got us
into places that we didn't know how to get out of. And Bob would say, "well Dave,
what do you think about how we ought to book this?" And I'd say, "gee I don't
know. What do you think?" And so we were where the two of us really didn't know how
to book it. And it was coming down the line the students were going to get up and gonna
ask for this. And my goodness, we were gonna to be in trouble. So Bob said, "here's
how I think," and he did it that way. And then he said, " the FASB has a hot
line. We'll call them up."
So he called them up. And he told them the
problem, and then there was a pause. And then the voice said, how are you guys gonna book
it? And Bob told them, and they said "sounds good to us." So that's how we book
it.
So what we're talking about is not easy It is extremely
difficult; Clearly it's difficult for the students. They're in there, but when we talk about ambiguity we really mean it. I mean, they're out there no matter how bright they are.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA020
The Business Activity Model at the University of Virginia:
Distributing Practice on a Given Task
In general, compared to distributing practice sessions on a given
task over time, massing practice or study sessions on the to-be-learned procedures or
information produces better short-term performance or recall of that procedure or
information, but markedly inferior long-term performance or recall.
Bjork (1994, p. 190)
Metacognition: Knowing about knowing
With blocked scheduling, each task is learned separately, and learning one task is
completed before the trainee moves to another task. With random scheduling, the tasks are
intermixed during acquisition . . . .Blocked scheduling always produced a faster rate of
learning during acquisition, but, regardless of the type of text scheduling, random
scheduling during acquisition resulted in the best performance at retention testing.
Healy and Sinclair (1996, p. 531)
Memory: Handbook of perception and cognition
You've got to understand, this was very
innovative. We don't try to cover a topic in one sitting, or three days in a row. We've
got seven years of this case. We're going to cover taxes for seven years. We don't have to
teach all the tax chapter in the first year . . . we have to teach enough to get them
through with how to book a loss. And then the company does turn it around in about the
third year --- it's just a little early. But now they are going to have to do deal with
those tax issues. So it's like peeling an onion. We go deeper and deeper into these taxes.
And I had the pleasure one of the first years I taught it of having one of our very bright
students come up to me in about the fifth year and say, you guys are going to make us do
taxes all seven years aren't you?
And I got to smile and tell them, you're going to do taxes all of
your life. Yes we did, all seven years.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
At this point let me say that a lot of our students don't listen
to us about documenting their work and organizing their work. And so when they come back .
. . this is great, I love this part! I go back in September when they return from
internships when we introduce year 4. We don't go back and redo everything for them, and
they need all the stuff from the first three years of the case. And they didn't organize
it, and they're dead, ok? They learn very quickly by year 5 and 6 how to organize a set of
work papers to support the financial statements.
Anthony Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
Whereas traditional Intermediate Accounting courses take up one text book topic at a
time and then move on to the next topic over the entire two or three semester course
coverage, the BAM model takes up nearly all topics repeatedly across two semesters. In
each of the seven years of accounting for the business, the same problems arise each year
in adjusting for accruals, estimating bad debts, valuing inventories, filing corporate tax
returns, etc.
UVA030
The Business Activity Model at the University of Virginia:
Varying the Conditions of Practice
It has now been demonstrated in a variety of ways, and with a
variety of motor, verbal, and problem-solving tasks, that introducing variation and/or
unpredictability in the training environment causes difficulty for the learner but
enhances long-term performance --- particularly the ability to transfer training to novel
but related task environments.
Bjork (1994, p. 189)
Metacognition: Knowing about knowing
How's this thing delivered? Ok, and I forgot to
answer your other question. I'll answer the other one . . . how the students react? First
semester, remember, you're destroying their model, their learning model, with a sledge
hammer. Ok? And so for the first two thirds of the semester, cuz I'm Satan! I'm a little
bit worse than Dave --- my students sit there like this . . . great body language . . .
for two thirds of the semester. Year 1, it's like pulling teeth.
Year 2: they finally realize that it's not going away. Ok? And
that you mean it, and that they have to play, and they start playing. Now where the real, that
real, hook comes in is when they go away for internships --- that's the summer between
the Intermediate 1 and Intermediate 2 semesters --- cuz that's where we place internships
at Virginia. After their internships, students come back and say "this is what really
happens!" Afterwards, in Intermediate 2, they've been revitalized. And they've bought
into BAM. And after that it's no problem
.
Anthony Catanach, University of
Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA040
The Business Activity Model at the University of Virginia:
Providing Contextual Interference
In Mannes and Kintsch’s (1987) experiment, for example, subjects had
to learn the content of a technical article (on industrial uses of microbes) after having
first studied an outline that was either consistent with the organization of the article
or inconsistent with that organization (but provided the same information in either case).
The inconsistent condition impaired subjects’ verbatim recall and recognition of the
article’s content (compared to the consistent condition), but facilitated performance
on tests that required subjects to infer answers or solve problems based on their general
understanding of the article’s content.
Bjork (1994, p. 190)
Metacognition: Knowing about knowing
And so they're seeing old and new, and I think
that's a pretty powerful teaching tool as well. A student discovered a problem in our
case. And what had happened is this, we had been taking a tax deduction for write-offs to
the bad debt. One of our students noted this --- it happened in my class. It happened in
my class naturally, the new guy on the faculty. And he said, Dr. Catanach, we just got out
of Ms. Jones class and she said that you couldn't take a deduction for that until it was
actually written off, you couldn't take it, you know, because you are using the allowance
method. I'm like, yeah, --- that sounds good to me. And then I said: "Well . .
. we'll get back to this later on."
And so I ran into my colleagues --- David and I and Bob --- we
had this debriefing and they went, yeah. Yes, so how are we going to do this, how are we
going to fix this? And so we went back to the class the next day; we said, don't worry
about the bad debt tax write-off. That's our position. We took a really weak defensive
position, and this is like year 2 of the case when this student pointed this out. And so
we let the mistake lie until year 6 of the case in the second semester. And in year 6 what
we did is we injected an IRS audit which would allow the IRS to come in and find the tax
error in year 2, and then students had to deal with a correction of an error issue. We
didn't have that in the original case. So then we put correction of an error assignment in
there. And if that wasn't enough, we had a stock split as well
.
Anthony
Catanach, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA050
The Business Activity Model at the University of Virginia:
Reducing Feedback to the Learner
. . . reducing the frequency of feedback makes life more
difficult for the learner during training, but can enhance post-training performance.
Bjork (1994, p. 191-192)
Metacognition: Knowing about knowing
. . . performance assessment during the acquisition phase of training is not a
reliable indicator of training efficiency and skill learning. Only post-training retention
and transfer testing, after an appropriate retention interval, can provide a true measure
of learning and performance and the effectiveness of any experimental manipulation
designed to enhance training efficiency.
Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition
. . . . described a study by Schooler and Anderson (1990) who showed that reducing the
number of feedback trials during the learning of a programming language decreased
acquisition performance but facilitated retention performance . . .
Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition
Professors Croll and Catanach made no mention of reduced feedback in Intermediate
Accounting before verus after adopting the BAM self-learning pedagogy. In both instances,
examinations were given at relatively frequent intervals and students receive feedback
about grade patterns to date in the course. Examinations were used to assess grades and
student study teams are not graded as teams. Also examination grading was not very
subjective on the types of examinations given in the courses. Frequent feedback and
relatively objective grading are probably major reasons why students did not give the
instructors low evaluations in the courses.
When attempts are made to improve long-term memory and performance by reducing feedback
frequency and more subjective grading near the end of each course, it is highly likely
that instructors will pay a heavy price in course evaluations by students. Also, keep in
mind that when students are taking multiple courses, they tend to concentrate on courses
more when tests are imminent. Frequency of examinations tends to increase attention given
to a course even though it may impair long-term performance where students must be graded
based upon final performance at the end of the course. It should be noted that if every
course based 100% of the course grade on the final examination and/or the end-of-term
project, it might unduly stress students at the end of the semester. There are of course
other alternatives. Examinations and project deadlines can be scheduled throughout the
course with the holding back of grading feedback until the entire set of things to be
graded are evaluated by course instructors. Students, however, despise delayed feedback
for a variety of reasons. Much remains to be researched on the tradeoff between feedback
in courses and long-term memory and performance. Another problem is that impacts of
feedback vary a great deal between students. Eric Jensen writes the following:
Does all this mean that external rewards are also good for the brain? The answer
is no. That’s because the brain’s internal reward system varies from one student
to the next. You’d never be able to have a fair system. How students respond depends
on genetics, their particular brain chemistry, and life experiences that have wired their
brains in a unique way. Rewards work as a complex system of neurotransmitters binding to
receptor sites on neurons. . . . Most teachers have found that the same external reward is
received differently by two different students.
E. Jensen (1998,
p. 65)
Teaching with the brain in mind
UVA060
The Business Activity Model at the University of Virginia:
Using Tests as Learning Events
And Landauer and Bjork (1978; see also Rea &
Modigliani, 1995) found that "expanding retrieval practice," in which
successive recall test are made progressively more difficult by increasing the time and
intervening events prior to each next test of some target information, facilitates
long-term recall substantially --- compared to the same number of tests administered at
constant (and easier) delays.
Bjork (1994, p. 192)
Metacognition: Knowing about knowing
Now, let me cover a couple things quickly here
and then I'll go back; one thing is this, I'm not really grading these students in terms
of their class participation and the group work. How I'm going to grade them is I'm going
to grade them the same way as I graded them before. I'm going to give them exams using old
CPA problems, but I changed some things slightly so that I agreed with the answers. But
for learning to learn, over two semesters, I'm going to give them six take home exams. I'm
fortunate at UVA --- we have an honor system, and it works. So I thought well, maybe some
schools won't have an honor system it may not work for them.
So I've tried take home exams in the summer school, where in
effect, they don't take it home they simply work on it in a three hour block while they
are in class. And I can watch them. And it works. The take home exams do not cover
material I've covered. They cover material we're going to cover. But it's fresh and new.
We give them the exam, they've got to go out, in some fashion, either with, well with
both, with the official pronouncements and with the intermediate text and find out the
answer to this. And sometimes to be nasty, we give them a problem having to do with a
forthcoming statement that hasn't been issued yet, so of course there's nothing in the
intermediate text so they have to read the pronouncement to figure out how to deal with
it.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA070
The Business Activity Model at the University of Virginia:
Impact on Teaching Evaluations
The tendency for instructors to be pushed toward training
programs that maximize the performance or evaluative reaction of their trainees during
is exacerbated by certain institutional characteristics that are common in real-world
organizations. First, those responsible for training are often themselves evaluated in
terms of the performance and satisfaction of their trainees during the training, or at the
end of the training. Second, individuals with the day-to-day responsibility for training
often do not get a chance to observe the post-training performance of the people they have
trained; a trainee’s later successes and failures tend to occur in settings that are
far removed from the original training environment, and from the trainer himself or
herself. It is also rarely the case that systematic measurements of the post-training
on-the-job performance are even collected, let alone provided to a trainer as a guide . .
.
Bjork (1994, p. 193)
Metacognition: Knowing about knowing
. . . performance assessment during the acquisition phase of training is not a reliable
indicator of training efficiency and skill learning. Only post-training retention and
transfer testing, after an appropriate retention interval, can provide a true measure of
learning and performance and the effectiveness of any experimental manipulation designed
to enhance training efficiency.
Healy and Sinclair (1996, p. 530)
Memory: Handbook of perception and cognition
A pedagogy that forces students to teach themselves or each other while relying less on
instructors may lead to lower teaching evaluations than a pedagogy where the teacher lives
up to student expectations of what a teacher should be doing inside and outside the
classroom. The BAM experiment is an exception. Professors Croll and Catanach had high
evaluations under traditional and under BAM experiment where they eliminated lectures and
forced students to learn on their own. It is not clear just why their evaluations did not
plummet. A major factor is that students had frequent feedback about grades and seemed to
be pleased that they were actually learning. A second factor is that students were allowed
to seek help in learning from most any source and were assigned to teams that seemed to
aid in the learning process. During class periods, the instructors "hovered"
close enough to team deliberations and most likely did more teaching than they care to
admit (e.g., when students really got hung up and appeared to run into a blank wall or
proceeded down wrong pathways). Outside the classroom, the instructors were available for
help. Classes were relatively small (around 35 students) such that instructors could
provide help to individuals and groups of students. Croll and Catanach attribute part of
the success of their teaching evaluations to "working toward solutions
together."
So what we're talking about is not easy. It is extremely
difficult --- clearly it's difficult for the students. They're in there, but when we
talked about ambiguity we really meant it. I mean, they're in deep water no matter how
bright they are. They're out there swimming. Now that may be why we got good reviews ---
we were swimming with them, and we all got through this thing together.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
A recent message from a biology professor at Trinity University reads as
follows:
The July 24, 1998 Chronicle of Higher Education
has a very interesting Point of View by Paul Trout of Montana State. To quote from the
article:
Administrators can rebut that calumny by using evaluation forms that are more than
student-satisfaction surveys. Instead of asking students to rate the professor’s
"stimulation of interest," "concern for students," and impartiality in
grading" - categories that allow disgruntled students to make pinatas of their
professors - evaluation forms should ask whether the course was demanding, whether
performance standards were high,, whether the workload was challenging, whether the
grading was tough, whether the student learned a lot. Those kinds of questions make it a
little harder for students who resent a heavy workload or low grades to give spiteful
responses.
It would seem that the Trinity Course Evaluation form as a little of each sentiment:
satisfaction and "tough." When the current form was being designed, these two
"sentiments" where not on my mind. It might be curious to rewrite the evaluation
form from these two perspectives and see what we get.
The Trout article was about incivility in the classroom and what he called
"education lite." I found it an interesting piece.
Blystone in Texas
Robert V. Blystone, Ph.D. <RBLYSTON@Trinity.edu>
Professor of Biology
Trinity University
San Antonio, Texas 78212
210.736-7243 210.736-7229 FAX
UVA080
The Business Activity Model at the University of Virginia:
Lessons From the Dead Poets Society
The movie Dead Poets Society showed examples of why students recalled so much of
their learning. There were changes in location, circumstances, use of emotions, movement,
and novel classroom positions. We know that learners remember much more when the learning
is connected to a field trip, music, a disaster, a guest speaker, or a novel learning
location. Follow up with a discussion, journal writing, a project, or peer teaching.
E. Jensen (1998,
p. 110)
Teaching with the brain in mind
Just like in regular business, Jerry Loose (the
error-prone Controller of the company in the BAM case) doesn't write notes. You write his
notes for him; it's his notes. But you end up writing his notes for him to do his
{function--not sure of this word}. And so, after the third year is completed, what's done
is we've got correct balance sheets, correct income statements, correct cash flow
statements, correct notes. Now how I teach that on the third day is, the students come in,
in their groups and sit down. I'll take a group, I'll take the spokesperson for the group,
send them to the board and say; write up the balance sheet, you are there. Write up the
income statement, we're getting them to share, you get cash flow and you just sit there
and we'll talk about notes when I get there. And off they go.
Now again, chances are the person I picked wasn't the one that
wrote this. There is so much work in this. I mean, we are not talking about small case.
When Bob got into it, he got into it. And this is the case. It can't be done by any single
individual --- you have to all share in doing this thing. And then you have to help each
other work your way through it. And so the person that's up doing a presentation is
getting coached from the sidelines by possibly the person that wrote it. Put this here,
put that here, here's my sheet. Do this. And once they're all up, I have whomever I called
on explain it all. Again with normally, with short coaching, they can explain the
important parts and with help from their team they can get all the parts of it.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA090
The Business Activity Model at the University of Virginia:
Find the Answers Where You May
Stated more broadly, the conditions of training need to be
constructed to reveal to the subject what knowledge and procedures are, and are not, truly
accessible under the types of conditions that can be expected to prevail in the
post-training environment. Some of the best ways to achieve that goal involve making life
seem more difficult for the learner.
Bjork (1994, p. 201)
Metacognition: Knowing about knowing
That's basically what we're doing. We're asking
people questions, they give us answers. Now, since it's a case, since we've been using it
for years now, one of the constant questions I get has to do with the fact that they say,
well, gee, my students would get the answers to this case and from the class before and it
would be useless. Well, you don't understand quite what we're doing here. I don't care
whether they get the answers from the class before them. In fact, we're telling these
people, you're working as a team, you're partners. Get the answers from your partner. All
the teams I have in class work for the same firm. Your group doesn't know the answer, ask
another group. See, as an academic I can't get over this. I won't ask questions of
anybody, it's cheating.
As an academic, for years I thought, gee, you're supposed to do
it all yourself. I got out into business, the more successful people asked others; that
knew this information. They picked up their phone, they didn't just make luncheon
appointments. They actually asked people about things. Why don't we have our students do
that? What am I doing then in class when we come to these questions?
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
UVA100
The Business Activity Model at the University of Virginia:
Importance of Context
It is in such special circumstances that the risk of having
contextualized training may be greatest. If we want people to respond optimally to
unanticipated novel conditions, such as emergencies and/or unique conditions of some other
type, the evidence summarized in this chapter suggests that we do not want to have trained
those people under fixed conditions.
Bjork (1994, p. 204)
Metacognition: Knowing about knowing
I want student groups to go away for a while and then come
back. What I want from you are questions. I want professional questions. I want a good
professional question from you, from each group, and I give them some hint as to how may
questions there are --- somewhere around ten questions. I also tell them an on-going hint
that Jerry has more problems as does normal business with errors of omission rather that
errors of commission. And so, take a look at it. You've had a review, you have had a
beginning accounting group. By then students are protesting that they don't know anything.
David Croll, University of Virginia
Transcribed from a videotape recording
Annual Meetings of the American Accounting Association
Dallas, Texas, August 20, 1997
Differences Between
Traditional Group
Learning and Cooperative Learning
In Appendix 2, I have reproduced an excerpt
that lists criteria for transforming group learning into cooperative learning. It
would appear that the BAM group learning is not technically cooperative learning in terms
of some of the Appendix 2 criteria.
Consider the Appendix 2 criteria listed
below:
- Cooperative learning groups are based on positive interdependence among
group members, where goals are structured so that students need to be concerned about
performance of all group members as well as their own.
- In cooperative learning groups, there is a clear individual accountability
where every student's mastery of the assigned material is assessed, each student is given
feedback on his or her progress, and the group is given feedback on how each member is
progressing so that the other groups' members know who to help and encourage. In
traditional learning groups, individual students are not often held individually
accountable for providing their share of the group's work and, occasionally, students will
"hitchhike" on the work of others.
Meeting the above two criteria is clearly difficult if this implies revealing each
others' examination performances and interim grades. Also it is not clear that most
groups work that way in the real world. At the University of Virginia, the BAM
instructors wander about the classroom observing groups in action. However, grading
criteria in the course apply only to individual performance and not upon group performance
or performances of other members of a group.
It is not at all clear how metacognitive processes are affected along a spectrum of
cooperation versus competition in learning. Clearly, colleges and universities tend
to be competitive in terms of grades. Virtually all courses have some type of
"curved" grading outcomes. Any course that is not somewhat competitive may
lose student motivation. For example, if everyone gets a A in Course A and 50% are
destined to get Cs in Course C, students most likely will strive harder in Course C.
The point here is that it is very difficult to disentangle cooperative/group
learning from reward systems.
From a metacognitive standpoint, long-term recall and feelings of knowing are more
likely to be greatly impacted by degree of effort and motivation, including competitively
inspired motivation. Also top students feel better about being distinguished as
being near the top of a class where not everyone in the class is at the top.
However, competition can also be destructive to aspirations and self confidence.
Broad generalizations about such matters are clearly hazardous.
Warnings: Suggestions for Future
Research and
Designs for Asynchronous Learning Networks
Warnings010
Fish Out of Water Risks
Years ago, R. Jensen (1970)
warned that relying on experiments from the behavioral sciences is analogous to living
with "fish out of water." These warnings apply to the experimental conclusions
of virtually all metalevel studies. In nearly every instance, the experiments have dealt
with memory and learning tasks far less complicated than coverage of an entire year of a
Intermediate Accounting. Many of the experiments focus on remembering names and places
rather than complex processes such as accounting for pensions, bad debt allowances,
depreciation, inventories, etc. Given the suspected success of BAM-type pedagogy, the time
is ripe for metacognitive and metamemory research in more complicated learning settings.
Warnings020
Contextualization Risks in Virtual Learning Worlds
One of the real problems of the BAM experiment is that it is confined mostly to
Intermediate Accounting topics. In the real world, advanced topics and complex contracting
issues surface that are well beyond the scope of intermediate accounting students.
Although the entire basis of the BAM pedagogy is to contextualize the learning environment
and force students to raise questions and find answers on their own, it is difficult to
prepare them for issues that are beyond the scope of the course or, for than matter,
beyond the scope of the entire college curriculum. The BAM pedagogy does, however, add
some topics not found in traditional Intermediate Accounting courses. Without ever having
a corporate tax course, students are required to fill out corporate tax forms across all
seven years of the business case. Secondly, they encounter some auditing issues prior to
having had an auditing course. By making students encounter and deal with issues for which
they have no prior background, the BAM pedagogy takes a giant stride away from excess
contextualization of traditional pedagogy where assignments typically are drills in
applying previously-learned concepts and procedures. However, only a small portion of the
plethora of novel real-world issues can be built into any case.
Warnings030
Misleading Program, Course, and Learning Module Evaluation Risks
I recently attended the Consortium of Liberal Arts College (CLAC) 1998 annual meetings.
This organization is comprised mainly of directors of information technology on liberal
arts campuses. These are the professionals in charge of assisting faculty in the design
and evaluation of newer learning technologies for synchronous and asynchronous programs,
courses, and learning modules. One of the most valuable sessions was entitled "How to
Evaluate Instructional Technology and How Technology Can be Used in Evaluating
Instruction." What struck me is the virtual consensus (among panelists and the
audience) that surveys and measurements of performance are untrustworthy except for
tabulations that are relatively easy to verify --- such as how many students own their own
computers, how many students use the Windows operating system, etc. Other statistics such
as the frequency of student visits to a web site or the number of messages in an email
listserv or chat room are viewed suspiciously since these statistics can be artificially
inflated in many ways.
In their opinions, anecdotal evidence and "stories" from faculty and students
were far more influential in administrative evaluations of programs, courses, and learning
modules than were formalized evaluations and measurement instruments. Obviously, such
presenters as Eleanor Lonske (Wellesley College), Diane Balestri (Vassar College), Phil
Harriman (The College of Wooster), Charles Christison (Beloit College), and Michael
Westfort (Connecticut College) admit that anecdotal evidence can be one-sided and
misleading, but far more dangerous in their eyes were quantitative studies that are
subject to too many intervening variables such as "times of day classes meet,"
"wordings of questions," "times and conditions of administering the
evaluation forms," and the "personalities of combinations of students in a given
course." In double blind studies (e.g., having the same instructor teach one class
using traditional materials and another class using newer technology aids) there is
seldom, if ever, a "clean experiment." Too many other uncontrolled variables
enter into such experiments. Also technologies change so fast that it is not clear that
extrapolations apply even one semester into the future. Meetings at the CLAC conference
did not mention Hawthorne effects. Hawthorne effects refer to distortions and possibly
non-sustaining effects of a treatment just because its newness captures more of an
individual's attentiveness. In double blind studies of the impact of technologies upon
learning, Hawthorne effects are particularly troublesome. Students are more apt to be more
attentive to newer technologies simply because they are "new" curiosities.
Positive results on learning impacts may not be sustaining, however, after the novelty and
curiosity factors decline with repeated use of the technology over time.
For more discussion of this topic, see
Appendix 1.
Warnings040
Metalevel Oversights in Learning Material Designs and Newer Pedagogy
Risks
In addition to the above concerns, I would add some metacognitive and metamemory
concerns for learning material designs. Students and faculty may have misleading feeling
of knowing (FOK) and satisfaction. Most formalized evaluations have a short-term
performance evaluations (e.g., examinations given during and at the end of a course).
Students tend to give higher evaluations to courses that did the following in their eyes:
- Made learning of complex material easier;
- Made it easier to find reference material for projects;
- Made learning more fun;
- Had new pedagogy (e.g., technologies) that captured attention because of its newness
(Hawthorne Effects);
- Had high frequency feedback so that students could adjust their study intensities
accordingly;
- Allowed greater topic coverage due to increased efficiencies of learning;
Fostered cooperative learning;
Made learning less frustrating and ambiguous;
Metalevel researchers in cognitive science view each of the above criteria with high
levels of suspicion. The problem is that the above criteria may lead to the following:
- Misleading metacognitive feeling of knowing (FOK) that does not carry over into real
world contextual variations and contextual complexity;
- Impaired long-term memory that would have been enhanced by more sweat, ambiguity,
frustration, and anxieties in the learning process;
- Impaired ability to adapt to changing contexts and creatively deal with variations in
challenges;
- Impaired abilities to face ambiguities;
Impaired abilities to face life's ultimate competitive challenges;
Impaired introspection;
In this paper, it is proposed that we may be designing our computer aided training and
education modules with the wrong criteria in mind. Perhaps it is wrong to always seek to
make learning easier, less frustrating, more collaborative, and more fun. In computer
aided learning, it is common to author in "rites of passage" that will not allow
students to move on to the next modules without demonstrating mastery of prior modules.
Perhaps rights of passage are not always ideal since reduced feedback sometimes leads to
improved metacognitive performance and long-term memory. Perhaps certain types of
frustrations aid learning. I see little value in the frustrations of computer failure and
slow Internet performance, but having to physically search among the stacks for hard copy
journals and books and having to locate references without the aid of keyword searches on
a computer may improve metacognitive performance.
Metalevel research findings have grave implications for our using emerging technologies
to create virtual learning worlds. The trend is toward making learning contextual. The
goal is to immerse students into simulations of reality so that learning is more
meaningful in the context to which it applies. However, any virtual world is a simplified
copy of a more complex world. Students who with strong feeling of knowing (FOK) in a
virtual world may be impaired by misleading feelings of knowing. Educators may be setting
them up for failure by enhancing FOK to a point where such feelings are dysfunctional
later on in life.
As mentioned already, the learner may be fooled by his or her own
successes during training. Manipulations such as blocking practice by subtask, providing
continuous feedback during training, and fixing the conditions of practice act like
crutches that artificially support performance during training. When those crutches are
absent in the post-training environment, performance collapses. The learner, however, will
typically lack the perspective and experience to realize that he or she has not yet
achieved the level of learning demanded by the post-training environment
Bjork (1994, p. 196)
Metacognition: Knowing about knowing
Warnings050
Excessive Collaboration Risks
Newer technologies such as email, electronic chat rooms, webcams, web documents,
telephony on the Internet, etc. make it increasingly easy, efficient, collaborative, and
effective to collaborate in training, education, and research. However, in increased
collaboration there may be too much of a good thing. To the extent that collaborations
reduce learning sweat and frustration there may be metacognitive impairments. There may
also be misleading feelings of knowing arising from such collaborations. Which parties in
the collaborations really had the "knowing" part of it? Collaborations may be
taking some important subjective experience out of the learning process.
Individuals who have illusions of comprehension or competence
pose a greater hazard to themselves and others than do individuals who correctly assess
that they lack some requisite information or skill. The reading we take of our own state
of knowledge determines whether we seek further study or practice, whether we volunteer
for certain jobs, whether we instill confidence in others, and so forth. In general, then,
as argued by Jacoby,
Bjork, and Kelly (1993), it is as important to educate
subjective experience as it is to educate objective experience.
Bjork (1994, p. 194)
Metacognition: Knowing about knowing
Conclusion
My colleagues and students will be surprised that I wrote this document. I am viewed as
a long-time advocate of computer aided learning technologies. However, this document is
entirely consistent with my emerging "gut feel" that many of the things I have
tried are not the best for my students. Across the years of experimenting with newer
technologies, I discovered the hard way that my masses of electronic transparencies, slide
shows, and similar lecture aids were likely impairing rather than aiding my lectures. As I
drifted away from the lecture pedagogy toward greater reliance upon an asynchronous online
pedagogy, I began to suspect that I was making asynchronous learning too easy by giving my
students instant access to experts (via audio, video, and searchable transcriptions) and
digitized literature databases on CD-ROMs and in server files. My one feeling of
satisfaction in the past year is that I no longer seek to make my courses easier for
students. I assign tougher term projects that reduce rather than enhance feedback
frequency by making students be more introspective and creative. To their consternation,
my grading is very subjective. In my latest technology experimentation, my students
express greater frustrations with course difficulty. Perhaps I wrote this metacognition
document to justify my gut feel that I am doing some things better when my course
evaluations decline.
My "discovery" of metacognitive and metamemory research strengthens my
worries and concerns about how we are designing our computer aided training and education
materials. It is terribly frustrating since these research findings destroy some of the
comparative advantages of emerging educational technologies. Computers can aid in
virtually all of Gardner's types of learning. For example, there are real comparative
advantages in creating contextualized virtual learning worlds, showing animated and
sequences of graphical images, increasing feedback frequency, making learning easier,
making learning more fun, making learning more collaborative, networking students with
experts, and making literature searches easier and more efficient. I
am not promoting reduced experimenting and implementing paces in computer and networking
technologies in education. What I am advocating is that the metacognitive principles
be programmed into these technologies even though doing so may weaken short-term learner
satisfaction with the program, course, and/or learning modules.
My "discovery" of the metacognitive and metamemory research strengthens my
worries and concerns about how we evaluate our programs, courses, and learning modules.
The frustrating part of metacognitive and metamemory criteria, when applied in higher
education, is that learning performance cannot be evaluated prior to graduation or even in
the earliest years of post-graduate work and study. Evaluating too soon may serve to put
too much weight on the wrong criteria (e.g., satisfaction) that are short sighted from a
metacognition standpoint. Our students may, thereby, be graduating with that misleading
FOK that impairs metacognitive performance in their longer-term futures. But then how many
of us in retrospect, years after graduation, have increased our regards for our toughest
and most frustrating professors? Most likely these are the cussed professors buried
deepest in metamemory.
In my viewpoint the above findings also have relevance to the findings of another
AECC-funded experiment called the Project Discovery (PD) project at the University of
Illinois at Urbana-Champaign. That study took a slightly different approach across
multiple accounting courses to study the effects of the importance of having students
learn from "complex, ill-structured, ambiguous problems and cases similar to those
found in practice" as reported by Stone and Shelly (1997, p. 26). Stone and Shelly repeatedly
stress that their research notes the impact of such factors on learning but make no
attempt to attribute causality. I speculate that some of the causal factors are
metacognitive. Even though my paper focused on the BAM Program at the University of
Virginia, I think my conclusions extend to the PD Program at the University of Illinois.
The Where's My Professor Game at Brigham Young University
Question
Why can this innovation probably be used only once for the full effect?
Hint: In subsequent terms the basic approach can be used although
students will by then know that the person in front is being paid to guide
learning.
"The Antiprofessor Speaks Out," by Kerry Soper, Chronicle of Higher
Education, The Chronicle Review (from the Chronicle of Higher
Education), December 5, 2008 ---
http://chronicle.com/weekly/v55/i15/15b02001.htm?utm_source=at&utm_medium=en
Now that the old teaching model of "sage on the
stage" has finally given way to the more progressive "guide at the side,"
academe is ready for another paradigm shift. In my classes, I have adopted a
philosophy that I call "peer at the rear." Here's how I break down and then
rebuild students' expectations of a healthy student-instructor relationship.
I start the first day of the semester at the back
of the classroom — literally. Students have no idea that I'm the professor.
I pull this off by wearing clothes from Old Navy, sporting a backward
baseball cap, and texting nonsense on a cellphone. If anyone tries to talk
to me, I am prepared with gossip about misbehaving celebrities or new
alternative-music bands.
When the real prof (the old me) doesn't show up
after about 15 minutes, I — as the pretend student — go ballistic. I loudly
"diss" the administration, the academic system, and myself (the outdated
teacher-me) and make a big show of riffling through the textbook and calling
it a load of you-know-what before tossing it into the garbage can.
After I cool down a bit, some of the students
usually start to leave. That's when I casually mention that I saw some
"cruddy syllabus or something" up at the front of the class. Of course they
dutifully grab the paper before heading off to their next old-fashioned
lecture. To further undermine the credibility of the phantom, old-school
professor in their eyes, I intentionally make this syllabus as obtuse,
incomplete, and condescending as possible.
At the start of the next class (while still
disguised as a hip, slouching student), I call out something like, "Well,
dudes, it looks like Señor Soper's not gonna show; guess we'll just have to
teach ourselves!" That's when the students really begin to take charge of
their own education. Granted, they mostly just read the newspaper or talk
about what they're going to eat when they get out of class, but I can see in
the way they carry themselves a new sense of ownership over their ideas and
lunch plans.
In the middle of this second class period, I crank
up some techno music on the sound system while doing a popping and locking
dance routine. In the middle of it, I write in giant letters on the board —
"Psych! I'm your instructor!" — and take off my baseball cap, revealing my
receding hairline. I can tell that some of the students are relieved, but I
keep them off balance by donning iPod earbuds, resuming my dance, and
pretending that I can't hear what they're saying.
If students get so frustrated that they start to
leave, I tone things down a bit and reveal the details of my
peer-at-the-rear philosophy. That includes doing an imitation of what my old
teaching persona might have done, had he been there. After getting a taste
of that pedagogical nerd, they seem to chill out a bit.
I lay the ground rules: They have to treat me as an
equal, not an authority figure or even a knowledgeable mentor. This includes
calling me by my first name (or a cool nickname like "Kerr Dawg" or "Super
Soper") and greeting me with some kind of groovester handshake or laid-back
fist bump. When that's settled, I throw up my hands, say, "Dudes, the class
is yours!," and watch as the magic unfolds.
Eventually some of the more alert students will
reluctantly organize themselves into study groups. This is a move in the
right direction; they're no longer relying on a self-inflated "professor" to
show them the way. But they're still full of predictably boring ideas, and
so I do my best to disrupt their discussions with postmodern Socratic
methods: walking around making annoying sounds; loudly interjecting Zen-like
non sequiturs into their conversations ("he who dealt it, smelt it"); or
standing behind someone while mouthing their words and mimicking their
posture.
To get things going on especially slow days, I do
have to facilitate a bit, but I like to keep it loose and open-ended. I
might show some music-videoclips and maybe a segment from The Colbert
Report, and then I'll just shrug my shoulders and say in a bored voice, "Wassup?"
This may irritate students who are still addicted to oppressive educational
methods, but that's my intention. Students need to be goaded to confront the
fallacies of the industrial/pseudo-educational complex, such as "grades
matter" and "professors know more than we do."
But educational misconceptions are so deeply
embedded that a large number of my young friends get frustrated with my
progressive methods; sometimes they even mount a campaign for a new
instructor or to get me fired. Right on! At least they're passionate about
an idea or cause — they're no longer passive robots.
Ultimately, though, my core objective is to become
students' buddy — their "homey," as it were. I try to achieve that rapport
by first turning their animosity toward deserving targets — anal parents,
stuffy professors, and faceless administrators. Then I build my own
egalitarian friendship with them in a number of relentlessly methodical
ways: following them around after class, texting them weird gossip about my
colleagues, forwarding them hilarious YouTube clips; and showing up at their
apartments to eat snacks and "crank some Halo" on the Xbox.
Many of the students resist those overtures,
probably because truly progressive changes always feel a little
uncomfortable at first. But by midsemester I usually manage to convert even
the most stalwart holdouts when I start undermining the university's "bogus"
grading system. If students insist on handing in essays, I mock traditional
evaluative judgments by writing nonsensical Beat poetry in the margins or by
marking every third sentence with a shiny kindergarten star. Sometimes I
even plagiarize random feedback from Wikipedia or SparkNotes, just to make a
point.
If students insist on taking a test, I adopt
Dadaistic strategies, like making them solve the kids' word jumble from the
Sunday comics page in a ridiculously short amount of time ("Go! You've got
12 seconds!") or assigning draconian grades based only on penmanship. The
latter traumatizes them until I make a big show of ripping up my grade book
and feeding it through a portable shredder. In the end, they get to choose
their own grade, of course. (Yes, it's usually an A, but as if they even
care at that point.)
Surprisingly, my student evaluations aren't as
stellar as you would expect. I get the occasional "You rock, dude," but for
the most part, my new peers seem too anxious to fully embrace my
antipedagogical persona. My colleagues, too, have been lukewarm about my
methods, some of them even holding special meetings to gripe about the way I
dress and the "chaos" I supposedly spread into their classrooms. And then,
of course, there are the ninnies in the administration building who don't
like the "grades" I give, the complaints they get from parents, and the way
I rap and beat-box loudly across the campus.
But I'm confident that eventually my new paradigm
will take hold and everyone will acknowledge that I was simply ahead of the
curve. I do hope I'll be vindicated before the special-action committee at
my university succeeds in firing me. But if I am fired, so be it. I'll be
remembered as the Galileo of my time, the "antiprofessor" bold enough to
give power back to the students, where it belongs. When our movement is
large enough, no one will be able to take us down (or, rather, make us stand
up). So as united peers, let us sit down in the rear and rock this old
school!n
Kerry Soper is an associate professor in the department of humanities,
classics, and comparative literature at Brigham Young University.
Jensen Comment
Perhaps without realizing it, perhaps Professor Soper's success with this is due
to metacognition that comes with self-learning tasks. A somewhat similar and
controversial pedagogy is used the the BAM approach to intermediate accounting
---
http://chronicle.com/weekly/v55/i15/15b02001.htm?utm_source=at&utm_medium=en
Years of experience with the BAM pedagogy shows that for some instructors it is
a disaster despised by students, although other instructors can like Anthony
Catanach can pull it off with award-winning success. It's very hard for teachers
not to teach!
Accounting Professors in Support of Online Testing That, Among Other
Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in
place of lecturing
"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August
29, 2007 ---
http://www.insidehighered.com/news/2007/08/29/e_test
Critics
of testing through the computer often argue that it’s
difficult to tell if students are doing their own work. It’s
also unclear to some professors whether using the technology
is worth their while.
A new study makes the argument that giving
electronic tests can actually reduce cheating and save
faculty time.
Anthony
Catanach Jr. and Noah Barsky, both associate professors of
accounting at the Villanova School of Business, came to that
conclusion after speaking with faculty members and analyzing
the responses of more than 100 students at Villanova and
Philadelphia University. Both Catanach and Barsky teach a
course called Principles of Managerial Accounting that
utilizes the WebCT Vista e-learning platform. The professors
also surveyed undergraduates at Philadelphia who took tests
electronically.
The
Villanova course follows a pattern of Monday lecture,
Wednesday case assignment, Friday assessment. The first two
days require in-person attendance, while students can check
in Friday from wherever they are.
“It never
used to make sense to me why at business schools you have
Friday classes,” Catanach said. “As an instructor it’s
frustrating because 30 percent of the class won’t show up,
so you have to redo material. We said, how can we make that
day not lose its effectiveness?”
The answer,
he and Barsky determined, was to make all electronically
submitted group work due on Fridays and have that be
electronic quiz day. That’s where academic integrity came
into play. Since the professors weren’t requiring students
to be present to take the exams, they wanted to deter
cheating. Catanach said programs like the one he uses
mitigate the effectiveness of looking up answers or
consulting friends.
In
electronic form, questions are given to students in random
order so that copying is difficult. Professors can change
variables within a problem to make sure that each test is
unique while also ensuring a uniform level of difficulty.
The programs also measure how much time a student spends on
each question, which could signal to an instructor that a
student might have slowed to use outside resources.
Backtracking on questions generally is not permitted.
Catanach said he doesn’t pay much attention to time spent on
individual questions. And since he gives his students a
narrow time limit to finish their electronic quizzes,
consulting outside sources would only lead students to be
rushed by the end of the exam, he added.
Forty-five
percent of students who took part in the study reported that
the electronic testing system reduced the likelihood of
their cheating during the course.
Stephen
Satris, director of the Center for Academic Integrity at
Clemson University, said he applauds the use of technology
to deter academic dishonesty. Students who take these
courses might think twice about copying or plagiarizing on
other exams, he said.
“It’s good
to see this program working,” Satris said. “It does an end
run around cheating.”
The report
also makes the case that both faculty and students save time
with e-testing. Catanach is up front about the initial time
investment: For instructors to make best use of the testing
programs, they need to create a “bank” of exam questions and
code them by topic, learning objectives and level of
difficulty. That way, the program knows how to distribute
questions. (He said instructors should budget roughly 10
extra hours per week during the course for this task.)
The payoff,
he said, comes later in the term. In the study, professors
reported recouping an average of 80 hours by using the
e-exams. Faculty don’t have to hand-grade tests (that often
being a deterrent for the Friday test, Catanach notes), and
graduate students or administrative staff can help prepare
the test banks, the report points out.
Since tests
are taken from afar, class time can be used for other
purposes. Students are less likely to ask about test results
during sessions, the study says, because the computer
program gives them immediate results and points to pages
where they can find out why their answers were incorrect.
Satris said this type of system likely dissuades students
from grade groveling, because the explanations are all there
on the computer. He said it also make sense in other ways.
“I like that
professors can truly say, ‘I don’t know what’s going to be
on the test. There’s a question bank; it’s out of my
control,’ ” he said.
And then
there’s the common argument about administrative efficiency:
An institution can keep a permanent electronic record of its
students.
Survey
results showed that Villanova students, who Catanach said
were more likely to have their own laptop computers and be
familiar with e-technology, responded better to the
electronic testing system than did students at Philadelphia,
who weren’t as tech savvy. Both Catanach and Satris said the
e-testing programs are not likely to excite English and
philosophy professors, whose disciplines call for essay
questions rather than computer-graded content.
From a
testing perspective, Catanach said the programs can be most
helpful for faculty with large classes who need to save time
on grading. That’s why the programs have proven popular at
community colleges in some of the larger states, he said.
“It works
for almost anyone who wants to have periodic assessment,” he
said. “How much does the midterm and final motivate students
to keep up with material? It doesn’t. It motivates cramming.
This is a tool to help students keep up with the material.”
August 29, 2007 reply from Stokes, Len
[stokes@SIENA.EDU]
I am also a strong proponent of active learning
strategies. I have the luxury of a small class size. Usually fewer than 30
so I can adapt my classes to student interaction and can have periodic
assessment opportunities as it fits the flow of materials rather than the
calendar. I still think a push toward smaller classes with more faculty face
time is better than computer tests. One lecture and one case day does not
mean active learning. It is better than no case days but it is still a
lecture day. I don’t have real lecture days every day involves some
interactive material from the students.
While I admit I can’t pick up all trends in grading
the tests, but I do pick up a lot of things so I have tendency to have a
high proportion of essays and small problems. I then try to address common
errors in class and also can look at my approach to teaching the material.
Len
Bob Jensen attempts to make a case that self learning is more effective
for metacognitive reasons ---
http://faculty.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob
Grinaker, and Noah Barsky.
Bob Jensen's threads on "Online Education Effectiveness and Testing" are
at
http://faculty.trinity.edu/rjensen/Assess.htm#OnlineOffCampus
Bob Jensen's threads on the myths of online education are at
http://faculty.trinity.edu/rjensen/000aaa/thetools.htm#Myths
Acknowledgement:
I want to thank Paula
Hertel, Professor and Chair of the Department of Psychology,
Trinity University. Dr. Hertel's specialty is memory and cognition research. She
remembered to give me some good leads for this document. She also reviewed the first
draft. All errors and shortcomings that remain are mine.
Appendix 1
Why Aren't Stories Good Food?
Email Messages About Evaluation Criteria and Processes
Email message from Bob Jensen to the AECM on June 28, 1998
My question is why aren't education researchers, journal editors, and referees
listening to the experts about evaluating programs, courses, and learning modules? Why do
we put up such hurdles for publication of "stories?"
This weekend I attended the Consortium of Liberal Arts College (CLAC) 1998 annual
meetings held on the Trinity University Campus. This organization is comprised mainly of
directors of information technology on liberal arts campuses. These are the professionals
in charge of assisting faculty in the design and evaluation of newer learning technologies
for synchronous and asynchronous programs, courses, and learning modules. One of the most
valuable sessions was entitled "How to Evaluate Instructional Technology and How
Technology can be Used in Evaluating Instruction." What struck me is the virtual
consensus (among panelists and the audience) that surveys and measurements of performance
are untrustworthy except for tabulations that are relatively easy to verify such as how
many students own their own computers, how many students use the Windows operating system,
etc. Other statistics such as the frequency of student visits to a web site or the number
of messages in an email listserv or chat room are viewed suspiciously since these
statistics can be artificially inflated in so many ways and say nothing about quality and
content.
In their opinions, anecdotal evidence and "stories" from faculty and students
were far more influential in administrative evaluations of programs, courses, and learning
modules than were formalized evaluations and measurement instruments. Obviously, such
presenters as Eleanor Lonske (Wellesley College), Diane Balestri (Vassar College), Phil
Harriman (The College of Wooster), Charles Christison (Beloit College), and Michael
Westfort (Connecticut College) admit that anecdotal evidence can be one-sided and
misleading, but far more dangerous in their eyes were quantitative studies that are
subject to too many intervening variables such as "times of day classes meet,"
"wordings of questions," "times and conditions of administering the
evaluations forms," and the "personalities of combinations of students in a
given course."
In double blind studies (e.g., having the same instructor teach one class using
traditional materials and another class using newer technology aids) there is seldom, if
ever, a "clean experiment." Too many other uncontrolled variables enter into
such experiments. Also technologies change so fast that it is not clear that
extrapolations apply even one semester into the future. Meetings at the CLAC conference
did not mention Hawthorne effects, but these also tend to distort evaluations of programs,
courses, and learning modules. Hawthorne effects are discussed in http://www.trinity.edu/~rjensen/245glosf.htm#Hawthorne
Hawthorne effects tend to bias student evaluations upward in favor of newer technologies.
Maybe I am venting my weariness of having to referee my 23rd paper focused
upon quantitative comparisons of distance education versus on-site courses. I'd rather
compare the stories from the instructors and students. Are we imposing the statistical
measurement criteria upon authors merely to satisfy some questionable biases of our
refereeing process? Are our referees listening to the experts?
The other message from the CLAC experts is that evaluations should always be focused
upon programs and not technologies per se.
Bob
Professor Robert E. Jensen (Bob) http://www.trinity.edu/~rjensen
Jesse H. Jones Distinguished Professor of Business Administration
Trinity University, San Antonio, TX 78212-7200
Voice: 210-736-7347 Fax: 210-736-8134
Email rensen@trinity.edu
Reply from Charles Walton
[ cwalton@usa.net ]
Bob,
Yesterday, Saturday, I finished a year long engagement
with a Board of Trustees of a college developing and implementing a new Board Governance
system.
The major time drain in developing new budgeting and
fiscal control system, was not restructuring Board budgeting, but was related dealing with
their problems in instructional and non-instruction IS functions. The Board has invested a
great deal of the taxpayers and students’ money in information technology with little
data to support that investment. There was no significant difference in student
satisfaction after implementation of technology and no significant difference in learning
outcomes between sections with and without the technology. The amounts requested were
staggering and those underrepresented or didn’t include maintenance.
On the administrative side, the systems are so poor that
the IS vendor holds the school hostage. The costs for a Datatel or PeopleSoft extrication
are totally out of proportion with the rest of the budget. The costs of salaries in
phasing out the old vendor, the new vendor transition project manager, instructional
computing head, and new CIO will be more than twice the costs of the President, VP of
Instruction, VP for Student Affairs and VP for Business Affairs for at least the next two
years.
I really feel for the Trustee who has to deal with these
IS requests. I shouldn’t have to vent, it made for an interesting sabbatical with a
lot of billable hours.
As I prepare to return from sabbatical, I must admit that
I avoid IS issues at my college. My need to manage IS was satisfied years ago as an IS
head.
Bob, Trustees are desperate for good food. Mostly, what
college administrators feed them gives them heartburn. A word of praise for non paid
Trustees who really attempt to do what is best for the college and the fanatic Trustees
who work very hard with little support to make it happen.
Charles Walton, Ph.D., CPA [ cwalton@usa.net ]
Gettysburg College
If you aren’t familiar with CMM and ISO 9000 you are
an IS dinosaur.
Reply from
Ailsa H. S. Nicholson [ ctiafm@uea.ac.uk ]
I would like to endorse Bob’s sentiments as regards telling the story.
Here in the UK the CTI (Computers in Teaching Initiative) have been encouraging
academics to ‘tell the story’ for the last nine years. Here at the CTI Centre
for Accounting Finance and Management (CTI-AFM) we have been encouraging lecturers in
accounting & business education to write ‘accounts’ of their use of
technology with classes for publlication on our journal ACCOUNT.
Those who do write for our journal and who contribute papers for our conference are
often very frank and reflective about the experience and share with others the downs as
well as the ups. Such reports are of course invaluable as it can prevent new adopters from
reinventing wheels or making avoidable errors.
‘Real evaluation’ is very long term and as you point out full of pitfalls and
unsurmountable problems. Also technology and software packages change or are updated so
often that any result of a long term study can be unhelpful in that they may refer to
products which have in the meantime been improved or changed. Telling the story is more
valuable to the practicioner .
Unfortunately as you point out it is difficult to get such reports published in
mainstream journals and certainly here in the UK and probably with you in the states
academics are strongly discouraged from any activity which does not lead to publication in
a mainstream journal. So it is an uphill struggle to ensure a good supply of reflectve
accounts - or stories - on the use of technology in teaching.
But CTI-AFM continues to seek these out and to persuade academics to write for our
journal or to contribute conference papers for our annual conference in April. Details of
both are on our www site - address below and we will shortly be putting ACCOUNT our
journal o’ Why Aren’t Stories Good Food? n-line so that colleagues in the US
will be able to view the experience of their UK colleagues.
Ailsa Nicholson
Ailsa H. S. Nicholson
CTI Accounting Finance and Management
School of Management
University of East Anglia
Norwich NR4 7TJ
Tel: 01603 592312
Fax: 01603 593343
E-mail: ctiafm@uea.ac.uk
http://www.mgt.uea.ac.uk/cti
Reply from Bob Dumouchel [ owl@callamerica.net ]
Hi Bob,
I just love this debate and as a vendor of self-study products I am anything but
impartial. These studies all seem to boil down to the same result.
"Format Does Not Make a Difference". Without conducting a study I would
contend that the difference when it is reported can be traced back to an improved student
perception of the course.
If schools want to increase the effectiveness of their courses they should market their
courses better and improve the student perception of the course after all "Perception
is more important that reality". If a student starts a course believing in it, they
will learn more. The strange thing about this is that at the university level the Profs
credentials are usually impressive to say the least. Yet if you read the average college
catalog you would never know it. When we have a product produced by someone with
impressive credentials you can believe that we make a big deal out of it.
Student perception needs to be managed, courses need to be marketed. If students
believe they need the course, if you create the want rather then stressing the need, the
results will follow. Professional quality instruction is of course an assumption in all of
this.
Good luck on your next study.
Bob Dumouchel
On With Learning Inc.
Phone (800) 272-0887 or (805) 481-0118
Fax (800) 508-0487 or (805) 481-0252
http://videoed.com
owl@callamerica.net
Appendix 3
The Emperor's Naked as He Can Be
Prior to one of my technology in education workshops, Bob Anthony sent me the following
questions that he expects me to answer in the workshop. Perhaps, more than me, you can
help him with some of these answers. Bob does not subscribe the aecm listserv, but you can
contact him at Robert N. Anthony at RNAnthony@valley.net
He's deadly serious about these questions. Bob is one of our most successful book
authors and has been retired from Harvard for many years. Nevertheless, he still has
a keen mind and his honest bluntness trademark. Years ago in Denmark, Bob was the little
boy on the streets who yelled out" "The Emperor's not wearing any clothes."
These questions are fundamental and we only have a limited amount of time in my
workshops to debate such issues. If those of you attending the workshops will prepare in
advance, we will try to take up these questions as time permits.
Some of the issues he raises taken up in the above document. Other issues are taken up
in its much larger companion piece at http://WWW.Trinity.edu/~rjensen/255wp.htm
Bob's message is repeated below below:
**************************************************************************************************************************************************
Bob,
I read with great interest the material you sent on July 31 and look forward to your
session on August 16. The following comment relates to the "self directed
learning" version of "distance learning," as distinguished from
"asynchronous learning" within a school. The issues relating to each of these
topics are somewhat different.
First, there is in implication in much of the material you quote that there is
something new about "self directed learning". Actually, the descriptions I have
seen aren’t much different from those of correspondence courses that have been
offered for many years. Some correspondence schools prosper, which indicates that they are
doing something that customers want. The inclusion of sound
is new, but what else?
It seems clear that interactive courses will not succeed. The effort involved in
responding to student queries is too great, especially in view of the difficulty of
obtaining money for these responses. Do you agree?
(Of course, the publisher must be prepared to correct errors.)
Many of the people you quote do not appreciate the difficulty of identifying
worthwhile; programs. This is an important function provided
by the successful textbook companies. I was consulting accounting editor
for Richard D. Irwin for many years, and read all the accounting manuscripts submitted to
them. Only a small fraction were worth publishing. There is a difference between the great
teacher and the great author. Bill Paton was not a leading textbook author. Although the
Irwin people were pessimistic, they published a text by [XXXXX} as a favor to me; it
didn’t succeed.
Therefore, the self-directed learning programs published
by universities are unlikely to do well. The university editors are
amateurs. Moreover, the value of a university "brand name" is not high in
accounting textbooks, and I don’t see why it should be higher on an Internet product.
Many mediocre courses will nevertheless appear on the Internet because the cost is so low.
(There is a company today that will scan a manuscript and publish it on the Internet,
without even commenting on its quality, for $500.) .
You mention corporate programs. My impression is that corporations develop successful
training programs, such as how to process information in a new purchase / accounts-payable
system; but training is quite different from education.
Colleges and universities focus on education.
I do believe that a few companies will publish good programs and therefore they will
before long dominate the market. Probably they will be conventional publishers because of
their editorial and marketing expertise. Maybe there will be a few new companies focusing
exclusively on the Internet.. You mention Knowledge Universe,
a company started by Milliken. I am familiar with UOL publishing. It’s too early to
tell who will emerge as the leaders, corresponding to the publishers of conventional
texts.
Arriving at sound conclusions to
issues like the above is important to me!
I am trying to make a version of my Essentials of Accounting succeed on the Internet,
on CD-ROM, or both. I hope some of the above topics will be discussed in the seminar.
. Bob Anthony.
"Critical Thinking: Why It's So Hard to Teach," by Daniel T.
Willingham ---
http://www.aft.org/pubs-reports/american_educator/issues/summer07/Crit_Thinking.pdf
Also see Simorleon Sense ---
http://www.simoleonsense.com/critical-thinking-why-is-it-so-hard-to-teach/
“Critical thinking is not a set of
skills that can be deployed at any time, in any context. It is a type of
thought that even 3-year-olds can engage in—and even trained scientists
can fail in.”
“Knowing that one should think
critically is not the same as being able to do so. That requires domain
knowledge and practice.”
So, Why Is Thinking Critically So
Hard?
Educators have long noted that school attendance and even academic
success are no guarantee that a student will graduate an effective
thinker in all situations. There is an odd tendency for rigorous
thinking to cling to particular examples or types of problems. Thus, a
student may have learned to estimate the answer to a math problem before
beginning calculations as a way of checking the accuracy of his answer,
but in the chemistry lab, the same student calculates the components of
a compound without noticing that his estimates sum to more than 100
percent. And a student who has learned to thoughtfully discuss the
causes of the American Revolution from both the British and American
perspectives doesn’t even think to question how the Germans viewed World
War II. Why are students able to think critically in one situation, but
not in another? The brief answer is: Thought processes are intertwined
with what is being thought about. Let’s explore this in depth by looking
at a particular kind of critical thinking that has been studied
extensively: problem solving.
Imagine a seventh-grade math class immersed in
word problems. How is it that students will be able to answer one
problem, but not the next, even though mathematically both word problems
are the same, that is, they rely on the same mathematical knowledge?
Typically, the students are focusing on the scenario that the word
problem describes (its surface structure) instead of on the mathematics
required to solve it (its deep structure). So even though students have
been taught how to solve a particular type of word problem, when the
teacher or textbook changes the scenario, students still struggle to
apply the solution because they don’t recognize that the problems are
mathematically the same.
Thinking Tends to Focus on a Problem’s
“Surface Structure”
To understand why the surface structure of a problem is so distracting
and, as a result, why it’s so hard to apply familiar solutions to
problems that appear new, let’s first consider how you understand what’s
being asked when you are given a problem. Anything you hear or read is
automatically interpreted in light of what you already know about
similar subjects. For example, suppose you read these two sentences:
“After years of pressure from the film and television industry, the
President has filed a formal complaint with China over what U.S. firms
say is copyright infringement. These firms assert that the Chinese
government sets stringent trade restrictions for U.S. entertainment
products, even as it turns a blind eye to Chinese companies that copy
American movies and television shows and sell them on the black market.”
With Deep Knowledge, Thinking Can
Penetrate Beyond Surface Structure
If knowledge of how to solve a problem never transferred to problems
with new surface structures, schooling would be inefficient or even
futile—but of course, such transfer does occur. When and why is
complex,5 but two factors are especially relevant for educators:
familiarity with a problem’s deep structure and the knowledge that one
should look for a deep structure. I’ll address each in turn. When one is
very familiar with a problem’s deep-structure, knowledge about how to
solve it transfers well. That familiarity can come from long-term,
repeated experience with one problem, or with various manifestations of
one type of problem (i.e., many problems that have different surface
structures, but the same deep structure). After repeated exposure to
either or both, the subject simply perceives the deep structure as part
of the problem description.
The Cambridge Handbook of Thinking and Reasoning
---
Click Here
The Miniature Guide To Critical Thinking Concepts & Tools ---
Click Here
Appendix 4
Update Message from Robert Bjork
I read your online article with great interest. You do an excellent job of making
concrete some very important points in your article, including, of course, some of the
points I tried to emphasize in my 1994 article.
The other important thing that comes through in your article is that computer-based
individualized learning techologies, for all of their potential, are not something
magical: They are tools that can be used in counterproductive and ill-advised ways as well
as in innovative and productive ways. Starting last year, for example, there was an
initiative at UCLA to have an internet site for every course at UCLA. That
initiative—well meaning, in my own opinion—has raised cries of alarm from
certain professors about infringement of intellectual property. Those concerns are
legitimate in a few cases, I think, but I (and I alone, I sometimes think) have been
concerned about the kind of issues you address in your article—namely, the potential
for such "hi-tech" resources to be used in ways that impede, rather than
promote, learning. Instructors now get day-to-day pressures from students to put most
everything on the web—overheads, outlines, lecture notes, etc. The web site then
becomes a kind of remedial device. Students decide that they can skip the lecture, or,
when they do attend, that they don’t need to take good notes, understand the lecture,
or ask questions when things are confusing, because they can (hopefully) get a repetition
on the course site.
There are, of course, creative ways to use a course site—ways that make the
learner an active participant in the learning process, but such exercises/materials to
enrich a course demand the professor’s time and energies and will not necessarily be
used or appreciated by students, who are prone to view such enriching exercises as an
additional course burden.
When I teach the graduate course on learning and memory this fall term, I would like to
consider using your article as one of the packet of course readings. I printed a copy from
the web site, but that copy is not one that would lend itself to being reproduced for the
course. Would you be willing to either send me a hard word-processing copy or attach one
to an email message?
A couple related articles of my own that appeared after the Bjork "Memory and
Metamemory Considerations" article are:
Bjork, R.A. Institutional impediments to effective training. (1994). In D. Druckman and
R.A.Bjork (Eds.), Learning, remembering, believing:
Enhancing human performance (pp.295-306). Washington, DC: National Academy Press.
Bjork, R. A. (in press). Assessing our own competence: Heuristics and illusions. In D.
Gopher and A. Koriat (Eds.), Attention and Peformance XVII. Cognitive Regulation of
Performance: Interaction of Theory and Application. Cambridge, MA: MIT Press. (41 pages)
I’ll have my assistant send copies of those articles to you.
Best regards,
RAB
Robert A. Bjork, Editor
PSYCHOLOGICAL REVIEW
Department of Psychology
UCLA
Los Angeles, CA 90095-1563
(310-825-7028; fax 310-206-5895)
"Are we over estimating remembering and underestimating learning?"
by Joe Proctor, Williams College
Memory And Learning: Recent Research
Main Category: Psychology /
Psychiatry Also Included In: Neurology /
Neuroscience Article Date: 08 Mar 2010 - 2:00 PST
http://www.medicalnewstoday.com/articles/181496.php
Are we over estimating remembering and
underestimating learning?
Current research by Nate Kornell, an assistant
professor of psychology at Williams College, and Robert A. Bjork of the
University of California, Los Angeles address this question and was recently
published in the Journal of Experimental Psychology.
In their paper titled A Stability Bias in Human
Memory: Overestimating Remembering and Underestimating Learning, Kornell and
Bjork write: "To manage one's own conditions of learning effectively
requires gaining an understanding of the activities and processes that do
and do not support learning."
In psychology, experts use the term metacognition
to talk about how people think about their own cognitive processes - in
essence, thinking about thinking.
To probe the way people think about their capacity
for remembering, Kornell and Bjork asked people to look at a list of words
and predict how well they would be able to remember the words after
subsequent periods of study and testing.
Their results led the researchers to the suggestion
that people are under confident in their learning abilities and
overconfident in their memories. That is, people failed to predict that they
would be able to remember more words after studying more - although in
reality, they learned far more -- instead basing their predictions on
current memory. Kornell and Bjork call this a "stability bias" in memory.
Kornell's work also has been published in
Scientific American, Psychological Science, Current Directions in
Psychological Science, and Applied Cognitive Psychology, among other
journals.
Appendix 5
The word "metacognition" arises once again.
"Assessing the Impact of Instructional Technology on Student
Achievement," by Lorraine Sherry, Shelley Billig, Daniel Jesse, and Deborah
Watson-Acosta, T.H.E. Journal, February 2001, pp. 40-43 --- http://www.thejournal.com/magazine/vault/A3297.cfm
Four separate simplified path analysis models were
tested. The first pair addressed process and product outcomes for class
motivation, and the second pair addressed school motivation. The statistically
significant (p < .05) results were as follows:
- Motivation was related to metacognition. The
relationship between class motivation and metacognition was slightly
stronger (R = .307, p < the relationship between school motivation and
metacognition (R = .282, p < .0001).
- The relationship between metacognition and inquiry
learning (Beta = .546, p < .0001) was stronger than the relationship
between metacognition and application of skills (Beta = .282, p <
.0001).
- The relationship between inquiry learning and the
student learning process outcome (Beta = .384, p = .001) was stronger than
the relationship between application of skills and the student learning
process outcome (Beta = -.055, not significant).
- The relationship between application of skills and
the student product outcome (Beta = .371, p = .004) was stronger than the
relationship between inquiry learning and the student product outcome
(Beta = .063, not significant).
Clearly, correlation does not imply causality.
However, when each of these elements was considered as an independent
variable, there was a corresponding change in associated dependent variables.
For example, there was a significant correlation between motivation and
metacognition, indicating that students' enthusiasm for learning with
technology may stimulate students' metacognitive (strategic) thinking
processes. The significant correlations between motivation, metacognition,
inquiry learning, and the student learning process score indicate that
motivation may drive increases in the four elements connected by the first
path. Similarly, the significant correlations between motivation,
metacognition, application of skills, and the student product score indicate
that motivation may drive increases in the four elements connected by the
second path.
Based on the significant correlations of the two
teacher measurements of student achievement with the student survey data,
these data validated the evaluation team's extension of the Developing
Expertise model to explain increases in student performance as a result of
engaging in technology-supported learning activities. Moreover, nearly all
students across the project met the standards for both the teacher-created
student product assessment and the learning process assessment. This indicates
that, in general, the project had a positive impact on student achievement.
Conclusions
These preliminary findings suggest that
teachers should emphasize the use of metacognitive skills, application of
skills, and inquiry learning as they infuse technology into their respective
academic content areas. Moreover, these activities are directly in line with
the Vermont Reasoning and Problem Solving Standards, and with similar
standards in other states. The ISTE/NETS standards for assessment and
evaluation also suggest that teachers:
- Apply technology in assessing student learning of
subject matter using a variety of assessment techniques.
- Use technology resources to collect and analyze
data, interpret results, and communicate findings to improve instructional
practice and maximize student learning.
- Apply multiple evaluation methods to determine
students' appropriate use of technology resources for learning,
communication and productivity.
Rockman (1998) suggests that "A clear assessment
strategy that goes beyond standardized tests enables school leaders,
policymakers, and the community to understand the impact of technology on
teaching and learning." RMC Research Corporation's extension of the
Sternberg model can be used to organize and interpret a variety of student
self-perceptions, teacher observations of student learning processes, and
teacher-scored student products. It captures the overlapping kinds of
expertise that students developed throughout their technology-related
activities.
One of the greatest challenges facing the Technology
Innovation Challenge Grants and the Preparing Tomorrow's Teachers To Use
Technology (PT3) grants is to make a link between educational technology
innovations, promising practices for teaching and learning with technology,
and increases in student achievement. We believe that this model may be
replicable in other educational institutions, including schools, districts,
institutions of higher learning, and grant-funded initiatives. However, to use
this model, participating teachers must be able to clearly identify the
standards they are addressing in their instruction, articulate the specific
knowledge and skills that are to be fostered by using technology, carefully
observe student behavior in creating and refining their work, and create and
benchmark rubrics that they intend to use to evaluate student work.
Appendix 6
Update in February 2003
Accounting Professors in Support of Online Testing That, Among Other
Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in
place of lecturing
"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August
29, 2007 ---
http://www.insidehighered.com/news/2007/08/29/e_test
Critics
of testing through the computer often argue that it’s
difficult to tell if students are doing their own work. It’s
also unclear to some professors whether using the technology
is worth their while.
A new study makes the argument that giving
electronic tests can actually reduce cheating and save
faculty time.
Anthony
Catanach Jr. and Noah Barsky, both associate professors of
accounting at the Villanova School of Business, came to that
conclusion after speaking with faculty members and analyzing
the responses of more than 100 students at Villanova and
Philadelphia University. Both Catanach and Barsky teach a
course called Principles of Managerial Accounting that
utilizes the WebCT Vista e-learning platform. The professors
also surveyed undergraduates at Philadelphia who took tests
electronically.
The
Villanova course follows a pattern of Monday lecture,
Wednesday case assignment, Friday assessment. The first two
days require in-person attendance, while students can check
in Friday from wherever they are.
“It never
used to make sense to me why at business schools you have
Friday classes,” Catanach said. “As an instructor it’s
frustrating because 30 percent of the class won’t show up,
so you have to redo material. We said, how can we make that
day not lose its effectiveness?”
The answer,
he and Barsky determined, was to make all electronically
submitted group work due on Fridays and have that be
electronic quiz day. That’s where academic integrity came
into play. Since the professors weren’t requiring students
to be present to take the exams, they wanted to deter
cheating. Catanach said programs like the one he uses
mitigate the effectiveness of looking up answers or
consulting friends.
In
electronic form, questions are given to students in random
order so that copying is difficult. Professors can change
variables within a problem to make sure that each test is
unique while also ensuring a uniform level of difficulty.
The programs also measure how much time a student spends on
each question, which could signal to an instructor that a
student might have slowed to use outside resources.
Backtracking on questions generally is not permitted.
Catanach said he doesn’t pay much attention to time spent on
individual questions. And since he gives his students a
narrow time limit to finish their electronic quizzes,
consulting outside sources would only lead students to be
rushed by the end of the exam, he added.
Forty-five
percent of students who took part in the study reported that
the electronic testing system reduced the likelihood of
their cheating during the course.
Stephen
Satris, director of the Center for Academic Integrity at
Clemson University, said he applauds the use of technology
to deter academic dishonesty. Students who take these
courses might think twice about copying or plagiarizing on
other exams, he said.
“It’s good
to see this program working,” Satris said. “It does an end
run around cheating.”
The report
also makes the case that both faculty and students save time
with e-testing. Catanach is up front about the initial time
investment: For instructors to make best use of the testing
programs, they need to create a “bank” of exam questions and
code them by topic, learning objectives and level of
difficulty. That way, the program knows how to distribute
questions. (He said instructors should budget roughly 10
extra hours per week during the course for this task.)
The payoff,
he said, comes later in the term. In the study, professors
reported recouping an average of 80 hours by using the
e-exams. Faculty don’t have to hand-grade tests (that often
being a deterrent for the Friday test, Catanach notes), and
graduate students or administrative staff can help prepare
the test banks, the report points out.
Since tests
are taken from afar, class time can be used for other
purposes. Students are less likely to ask about test results
during sessions, the study says, because the computer
program gives them immediate results and points to pages
where they can find out why their answers were incorrect.
Satris said this type of system likely dissuades students
from grade groveling, because the explanations are all there
on the computer. He said it also make sense in other ways.
“I like that
professors can truly say, ‘I don’t know what’s going to be
on the test. There’s a question bank; it’s out of my
control,’ ” he said.
And then
there’s the common argument about administrative efficiency:
An institution can keep a permanent electronic record of its
students.
Survey
results showed that Villanova students, who Catanach said
were more likely to have their own laptop computers and be
familiar with e-technology, responded better to the
electronic testing system than did students at Philadelphia,
who weren’t as tech savvy. Both Catanach and Satris said the
e-testing programs are not likely to excite English and
philosophy professors, whose disciplines call for essay
questions rather than computer-graded content.
From a
testing perspective, Catanach said the programs can be most
helpful for faculty with large classes who need to save time
on grading. That’s why the programs have proven popular at
community colleges in some of the larger states, he said.
“It works
for almost anyone who wants to have periodic assessment,” he
said. “How much does the midterm and final motivate students
to keep up with material? It doesn’t. It motivates cramming.
This is a tool to help students keep up with the material.”
August 29, 2007 reply from Stokes, Len
[stokes@SIENA.EDU]
I am also a strong proponent of active learning
strategies. I have the luxury of a small class size. Usually fewer than 30
so I can adapt my classes to student interaction and can have periodic
assessment opportunities as it fits the flow of materials rather than the
calendar. I still think a push toward smaller classes with more faculty face
time is better than computer tests. One lecture and one case day does not
mean active learning. It is better than no case days but it is still a
lecture day. I don’t have real lecture days every day involves some
interactive material from the students.
While I admit I can’t pick up all trends in grading
the tests, but I do pick up a lot of things so I have tendency to have a
high proportion of essays and small problems. I then try to address common
errors in class and also can look at my approach to teaching the material.
Len
Bob Jensen attempts to make a case that self learning is more effective
for metacognitive reasons ---
http://faculty.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob
Grinaker, and Noah Barsky.
Bob Jensen's threads on "Online Education Effectiveness and Testing" are
at
http://faculty.trinity.edu/rjensen/Assess.htm#OnlineOffCampus
Bob Jensen's threads on the myths of online education are at
http://faculty.trinity.edu/rjensen/000aaa/thetools.htm#Myths
Team Learning versus Lecture Learning
May 22, 2010 message from J. S. Gangolly
[gangolly@CSC.ALBANY.EDU]
Bob,
Team-based learning is an alternative to lecturing.
However, the students are tested individually as well as teams.
As I understand, each session starts with an individual quizz and
then the team discussion of problems (usually cases) and their solution.
Some references:
http://ctl.stanford.edu/Tomprof/postings/750.html
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1546479
http://www.cshe.unimelb.edu.au/pdfs/Team_problem.pdf
http://www.ecampus.com/bk_searchresult.asp?qtype=ISBN&qsearch=9781579220860
Jagdish
Accounting Professors in Support of Online Testing That, Among Other
Things, Reduces Cheating
These same professors became widely known for their advocacy of self-learning in
place of lecturing
"In Support of the E-Test," by Elia Powers, Inside Higher Ed, August
29, 2007 ---
http://www.insidehighered.com/news/2007/08/29/e_test
Critics
of testing through the computer often argue that it’s
difficult to tell if students are doing their own work. It’s
also unclear to some professors whether using the technology
is worth their while.
A new study makes the argument that giving
electronic tests can actually reduce cheating and save
faculty time.
Anthony
Catanach Jr. and Noah Barsky, both associate professors of
accounting at the Villanova School of Business, came to that
conclusion after speaking with faculty members and analyzing
the responses of more than 100 students at Villanova and
Philadelphia University. Both Catanach and Barsky teach a
course called Principles of Managerial Accounting that
utilizes the WebCT Vista e-learning platform. The professors
also surveyed undergraduates at Philadelphia who took tests
electronically.
The
Villanova course follows a pattern of Monday lecture,
Wednesday case assignment, Friday assessment. The first two
days require in-person attendance, while students can check
in Friday from wherever they are.
“It never
used to make sense to me why at business schools you have
Friday classes,” Catanach said. “As an instructor it’s
frustrating because 30 percent of the class won’t show up,
so you have to redo material. We said, how can we make that
day not lose its effectiveness?”
The answer,
he and Barsky determined, was to make all electronically
submitted group work due on Fridays and have that be
electronic quiz day. That’s where academic integrity came
into play. Since the professors weren’t requiring students
to be present to take the exams, they wanted to deter
cheating. Catanach said programs like the one he uses
mitigate the effectiveness of looking up answers or
consulting friends.
In
electronic form, questions are given to students in random
order so that copying is difficult. Professors can change
variables within a problem to make sure that each test is
unique while also ensuring a uniform level of difficulty.
The programs also measure how much time a student spends on
each question, which could signal to an instructor that a
student might have slowed to use outside resources.
Backtracking on questions generally is not permitted.
Catanach said he doesn’t pay much attention to time spent on
individual questions. And since he gives his students a
narrow time limit to finish their electronic quizzes,
consulting outside sources would only lead students to be
rushed by the end of the exam, he added.
Forty-five
percent of students who took part in the study reported that
the electronic testing system reduced the likelihood of
their cheating during the course.
Stephen
Satris, director of the Center for Academic Integrity at
Clemson University, said he applauds the use of technology
to deter academic dishonesty. Students who take these
courses might think twice about copying or plagiarizing on
other exams, he said.
“It’s good
to see this program working,” Satris said. “It does an end
run around cheating.”
The report
also makes the case that both faculty and students save time
with e-testing. Catanach is up front about the initial time
investment: For instructors to make best use of the testing
programs, they need to create a “bank” of exam questions and
code them by topic, learning objectives and level of
difficulty. That way, the program knows how to distribute
questions. (He said instructors should budget roughly 10
extra hours per week during the course for this task.)
The payoff,
he said, comes later in the term. In the study, professors
reported recouping an average of 80 hours by using the
e-exams. Faculty don’t have to hand-grade tests (that often
being a deterrent for the Friday test, Catanach notes), and
graduate students or administrative staff can help prepare
the test banks, the report points out.
Since tests
are taken from afar, class time can be used for other
purposes. Students are less likely to ask about test results
during sessions, the study says, because the computer
program gives them immediate results and points to pages
where they can find out why their answers were incorrect.
Satris said this type of system likely dissuades students
from grade groveling, because the explanations are all there
on the computer. He said it also make sense in other ways.
“I like that
professors can truly say, ‘I don’t know what’s going to be
on the test. There’s a question bank; it’s out of my
control,’ ” he said.
And then
there’s the common argument about administrative efficiency:
An institution can keep a permanent electronic record of its
students.
Survey
results showed that Villanova students, who Catanach said
were more likely to have their own laptop computers and be
familiar with e-technology, responded better to the
electronic testing system than did students at Philadelphia,
who weren’t as tech savvy. Both Catanach and Satris said the
e-testing programs are not likely to excite English and
philosophy professors, whose disciplines call for essay
questions rather than computer-graded content.
From a
testing perspective, Catanach said the programs can be most
helpful for faculty with large classes who need to save time
on grading. That’s why the programs have proven popular at
community colleges in some of the larger states, he said.
“It works
for almost anyone who wants to have periodic assessment,” he
said. “How much does the midterm and final motivate students
to keep up with material? It doesn’t. It motivates cramming.
This is a tool to help students keep up with the material.”
August 29, 2007 reply from Stokes, Len
[stokes@SIENA.EDU]
I am also a strong proponent of active learning
strategies. I have the luxury of a small class size. Usually fewer than 30
so I can adapt my classes to student interaction and can have periodic
assessment opportunities as it fits the flow of materials rather than the
calendar. I still think a push toward smaller classes with more faculty face
time is better than computer tests. One lecture and one case day does not
mean active learning. It is better than no case days but it is still a
lecture day. I don’t have real lecture days every day involves some
interactive material from the students.
While I admit I can’t pick up all trends in grading
the tests, but I do pick up a lot of things so I have tendency to have a
high proportion of essays and small problems. I then try to address common
errors in class and also can look at my approach to teaching the material.
Len
Bob Jensen attempts to make a case that self learning is more effective
for metacognitive reasons ---
http://faculty.trinity.edu/rjensen/265wp.htm
This document features the research of Tony Catanach, David Croll, Bob
Grinaker, and Noah Barsky.
Bob Jensen's threads on "Online Education Effectiveness and Testing" are
at
http://faculty.trinity.edu/rjensen/Assess.htm#OnlineOffCampus
Bob Jensen's threads on the myths of online education are at
http://faculty.trinity.edu/rjensen/000aaa/thetools.htm#Myths
Hi Yvonne,
For what it is worth, my advice to new
faculty is at http://faculty.trinity.edu/rjensen/000aaa/newfaculty.htm
One thing to remember is that the
employers of our students (especially the public accounting firms) are very
unhappy with our lecture/drill pedagogy at the introductory and intermediate
levels. They believe that such pedagogy turns away top students, especially
creative and conceptualizing students. Employers believe that
lecture/drill pedagogy attracts savant-like memorizers who can recite their
lessons book and verse but have few creative talents and poor prospects for
becoming leaders. The large accounting firms believed this so strongly that they
donated several million dollars to the American Accounting Association for the
purpose of motivating new pedagogy experimentation. This led to the Accounting
Change Commission (AECC) and the mixed-outcome experiments that followed. See http://accounting.rutgers.edu/raw/aaa/facdev/aecc.htm
The easiest pedagogy for faculty is
lecturing, and it is appealing to busy faculty who do not have time for students
outside the classroom. When lecturing to large classes it is even easier because
you don't have to get to know the students and have a great excuse for using
multiple choice examinations and graduate student teaching assistants. I always
remember an economics professor at Michigan State University who said that when
teaching basic economics it did not matter whether he had a live class of 300
students or a televised class of 3,000 students. His full-time teaching load was
three hours per week in front of a TV camera. He was a very good lecturer and
truly loved his three-hour per week job!
Lecturing appeals to faculty because it
often leads to the highest teaching evaluations. Students love faculty who
spoon feed and make learning seem easy. It's much easier when mom or dad
spoon the pudding out of the jar than when you have to hold your own spoon
and/or find your own jar.
An opposite but very effective pedagogy
is the AECC (University of Virginia) BAM Pedagogy that entails live classrooms
with no lectures. BAM instructors think it is more important for students to
learn on their own instead of sitting through spoon-fed learning lectures. I
think it takes a special kind of teacher to pull off the astoundingly successful
BAM pedagogy. Interestingly, it is often some of our best lecturers who decided
to stop lecturing because they experimented with the BAM and found it to be far
more effective for long-term memory. The top BAM enthusiasts are Tony Catanach
at Villanova University and David Croll at the University of Virginia. Note,
however, that most BAM applications have been at the intermediate accounting
level. I have my doubts (and I think BAM instructors will agree) that BAM will
probably fail at the introductory level. You can read about the BAM pedagogy at http://faculty.trinity.edu/rjensen/265wp.htm
At the introductory level we have what
I like to call the Pincus (User Approach) Pedagogy. Karen Pincus is now at the
University of Arkansas, but at the time that her first learning experiments were
conducted, she taught basic accounting at the University of Southern California.
The Pincus Pedagogy is a little like both the BAM and the case method
pedagogies. However, instead of having prepared learning cases, the Pincus
Pedagogy sends students to on-site field visitations where they observe on-site
operations and are then assigned tasks to creatively suggest ways of improving
existing accounting, internal control, and information systems. Like the BAM,
the Pincus Pedagogy avoids lecturing and classroom drill. Therein lies the
controversy. Students and faculty in subsequent courses often complain that the
Pincus Pedagogy students do not know the fundamental prerequisites of basic
accounting needed for intermediate and advanced-level accounting courses.
Two possible links of interest on the controversial Pincus Pedagogy are as
follows:
Where the Pincus Pedagogy and the BAM
Pedagogy differ lies in subject matter itself and stress on creativity. The BAM
focuses on traditional subject matter that is found in such textbooks as
intermediate accounting textbooks. The BAM Pedagogy simply requires that
students learn any way they want to learn on their own since students remember
best what they learned by themselves. The Pincus Pedagogy does not focus on much
of the debit and credit "rules" found in most traditional textbooks.
Students are required to be more creative at the expense of memorizing the
"rules."
The Pincus Pedagogy is motivated by the
belief that traditional lecturing/drill pedagogy at the basic accounting and tax
levels discourages the best and more-creative students to pursue careers in the
accountancy profession. The BAM pedagogy is motivated more by the belief that
lecturing is a poor pedagogy for long-term memory of technical details. What is
interesting is that the leading proponents of getting away from the
lecture/drill pedagogy (i.e., Karen Pincus and Anthony Catenach) were previously
two of the very best lecturers in accountancy. If you have ever heard either of
them lecture, I think you would agree that you wish all your lecturers had been
only half as good. I am certain that both of these exceptional teachers would
agree that lecturing is easier than any other alternatives. However, they do not
feel that lecturing is the best alternative for top students.
Between lecturing and the BAM Pedagogy,
we have case method teaching. Case method teaching is a little like lecturing
and a little like the BAM with some instructors providing answers in case wrap
ups versus some instructors forcing students to provide all the answers. Master
case teachers at Harvard University seldom provide answers even in case wrap
ups, and often the cases do not have any known answer-book-type solutions. The
best Harvard cases have alternative solutions with success being based upon
discovering and defending an alternative solution. Students sometimes
interactively discover solutions that the case writers never envisioned. I
generally find case teaching difficult at the undergraduate level if students do
not yet have the tools and maturity to contribute to case discussions.
Interestingly, it may be somewhat easier to use the BAM at the undergraduate
level than Harvard-type cases. The reason is that BAM instructors are often
dealing with more rule-based subject matter such as intermediate accounting or
tax rather than conceptual subject matter such as strategic decision making,
business valuation, and financial risk analysis.
The hardest pedagogy today is probably
a Socratic pedagogy online with instant messaging communications where an
instructor who's on call about 60 hours per week from his or her home. The
online instructor monitors the chats and team communications between students in
the course at most any time of day or night. Amy Dunbar can tell you about this
tedious pedagogy since she's using it for tax courses and will be providing a
workshop that tells about how to do it and how not to do it. The next scheduled
workshop precedes the AAA Annual Meetings on August 1, 2003 in Hawaii. You can
also hear Dr. Dunbar and view her PowerPoint show from a previous workshop at http://www.cs.trinity.edu/~rjensen/002cpe/02start.htm#2002
In conclusion, always remember that
there is no optimal pedagogy in all circumstances. All learning is
circumstantial based upon such key ingredients as student maturity, student
motivation, instructor talent, instructor dedication, instructor time, library
resources, technology resources, and many other factors that come to bear at
each moment in time. And do keep in mind that how you teach may determine what
students you keep as majors and what you turn away.
I tend to agree with the accountancy
firms that contend that traditional lecturing probably turns away many of the
top students who might otherwise major in accountancy.
At the same time, I tend to agree with
students who contend that they took accounting courses to learn accounting
rather than economics, computer engineering, and behavioral science.
Bob Jensen
-----Original
Message-----
From: Lou&Bonnie [mailto:gyp1@EARTHLINK.NET]
Sent: Thursday, January 16, 2003 5:03 PM
I am a beginning
accounting instructor (part-time) at a local community college. I am applying
for a full-time faculty position, but am having trouble with a question.
Methodology in accounting--what works best for a diversified group of
individuals. Some students work with accounting, but on a computer and have no
understanding of what the information they are entering really means to some
individuals who have no accounting experience whatsoever. What is the best
methodology to use, lecture, overhead, classroom participation? I am not sure
and I would like your feedback. Thank you in advance for your help.
Yvonne
February 5,, 2003 message from James Borden [james.borden@villanova.edu]
Bob,
I thought you might be interested in another
curriculum innovation that is taking place at Villanova, once again involving
Tony Catanach, along with Noah Barsky. Noah is one of our young professors at
Villanova who is an outstanding teacher, and has been committed to developing
the Business Planning Model (BPM) approach to teaching Management Accounting
for some time now. If you have any questions about the paper, feel free to
contact Noah at noah.barsky@villanova.edu
Thank you for continuing to support the BAM approach
to teaching Intermediate as well!
Jim
Note that Noah has a PDF file that he will probably send to you if you
request it from him.
February 5, 2003 message from Milt Cohen
-----Original Message-----
From: Milt Cohen, Accounting Instructor [mailto:uncmlt@JUNO.COM]
Sent: Wednesday, February 05, 2003 9:54 PM
A thought for the bright minded on this web site,
after watching an H & R Block TV commercial where they boast about getting
more refunds for taxpayers when they review their prior year's tax forms, do
you think that if they find that the taxpayer OWES money for a prior year
and......... if the taxpayer says, "hell no, I won't pay"
............are they obligated to report the short pay (tax owed) to the IRS?
Interesting letter in the current Journal of
Accountancy , Feb. 2003 issue on pages 11 and 12, a professor of
Accounting at Mesa State College in Grand Junction, Colorado, writes about how
poorly the current text book used in his college classes do not provide
explanation as to how transactions (of businesses) get recorded into the books
and records of a business. And that the emphasis is on financial statement
analysis. I can't help recalling how I (and my classmates) slaved over work
problems in accounting classes in the 1950s. And how I emphasis (in my
classes) the bookkeeping procedures. It's hard to realize that college
Accounting classes ignore the bookkeeping phase of the profession. Without a
bookkeeper, there is no basis for financial statement analysis.
Just a couple of thoughts.
Appendix 7
How the Brain Deals With Information Overload
"Attention Must Be Paid," by Scott McLemee, Inside Higher Ed, November 26,
2008 ---
http://www.insidehighered.com/views/2008/11/26/mclemee
Actually looking at the e-mail would no doubt have
informed me that it was a matter of paying a modest fee to some enterprising
soul, probably in the Cayman Islands. Instead, I deleted this message on the
basis of the subject line alone, along with a dozen other such
communications. Meanwhile, my eyeballs were unwittingly drawn to a video
loop of a woman screaming in terror – horrified at high credit card interest
rates, which she could reduce via a company that advertises with my e-mail
provider.
Then my cell phone emitted a short burst of music,
announcing that someone had just left a text message.
All par for the course, of course. (At least I
wasn’t driving.) The demands on our attention have now become a matter for
professional expertise: An organization for specialists, the
Information
Overload Research Group, was formally incorporated
as a nonprofit this summer and held its first conference in August. A
substantial
technical literature
on interruption now exists. And one
recent consideration of the world economic crisis
suggests it has been exacerbated by all the data now sloshing around the
globe: “We have far too much information today and that impedes our
decision-making abilities and throttles our ability to resolve crises.”
The weak link in the information age seems to be
our human hard-wiring. So one gathers from
The Overflowing Brain: Information Overload and the Limits of Working Memory
(Oxford University Press) by Torkel Klingberg, who is
a professor of developmental cognitive neuroscience at the Stockholm Brain
Institute. A review of recent research on how attention and memory actually
function within our gray matter, it is a work of scientific popularization
rather than a handbook on how to minimize the cognitive drain of
distraction.
But there may be some advantage to knowing how the
systems in our heads actually operate – and it is Klingberg’s contention
that, in spite of everything, those systems may actually benefit from the
sometimes excessive demands our environments now place on our capacity to
process the data flux. The human brain itself has not changed much in either
anatomy or volume over the past 40,000 years. So at one level it seems
natural that we should experience a cognitive bottleneck in handling the
masses of information being hurled at us daily.
To simplify Klingberg’s already pared-down
analysis, we can distinguish between two
kinds of attention. One is controlled
attention: the directed effort to apply one’s concentration to a particular
task. The other is stimulus-driven attention, which is an involuntary
response to something happening in the environment. (You can tune out the
conversations going on around you in a restaurant. But if a waiter drops a
tray full of dishes, it is going to impose itself on your awareness.)
But it’s not as if these forms of attention are –
as it may seem – different manifestations of the same state of
consciousness: researchers have found from tests that the controlled and
stimulus-driven attention “seem fairly independent of one another,” says
Klingberg, which may mean “that there are different parts of the brain, or
different brain processes” involved in them.
Likewise, there is a distinction between the kind
of memory that allows you to recall an event from five years ago and a set
of information connected with a problem you are trying to solve. Your
recollections of yesteryear are part of long-term memory, which can be
mysteriously capacious. By contrast, there are definite limitations on how
much task-oriented data can be held in your “working memory.” (Evidently
there are grounds for debate among researchers over whether or not this is
the same as “short-term memory,” but we’ll just stick to Klingberg’s
preferred usage.)
As with the forms of attention, the distinction
between long-term and working memory corresponds to different processes
within the brain, occurring within different parts of its geography. But
there is evidence that (as you might expect) working memory and controlled
attention are closely related. People who score lower on tests for the
ability to retain information in their working memory tend to have more
difficulty in focusing attention on a complex task. “It might not come as
too much of a surprise,” says Klingberg, “to find that working memory
capacity correlates highly with reading comprehension.”
Klingberg reports that a two-year study in his lab
showed that it was possible to increase working-memory capacity: “children
who had done a certain type of computerized memory task, such as remembering
positions in a four-by-four grid and clicking a mouse button, improved at
other, noncomputerized types of working memory too.... We had shown that the
systems are not static and that the limits of working memory capacity can be
stretched.”
Further study suggested that this improvement also
corresponded to increased problem-solving skills. Our brains may still have
many of the same fundamental limitations as the Cro-Magnon model, but there
is also some degree of plasticity in how we can use and develop it.
Which brings us to Klingberg’s most surprising and
even counterintuitive suggestion. Multitasking often threatens to overload
the working memory. But at the same time, it’s clear that we can actually
manage it, at least to some degree – reading a newspaper while walking on a
treadmill, for example, and occasionally glancing up at the TV screen to see
what’s breaking on CNN.
“There is, fortunately, no research suggesting that
exposure to mentally more demanding or challenging situations impairs our
powers of concentration,” writes Klingberg. “Indeed, there is much that
points to the contrary: it is in situations that push the boundaries of our
abilities that we train our brains the most.”
But even if our basic ability to process
information is increasing, a growing “discrepancy between demand and
capacity” may account for the common sense of losing focus.
“You are very possibly 10 percent better at talking
on the phone while erasing spam today than you were three years ago. On the
other hand, the number of e-mails you receive per day has probably shot up
about 200 percent. There is, therefore, no contradiction between the feeling
that your abilities are inadequate and the improvement of those abilities.”
Well, that is some comfort – if not much. It’s been
said that the scarcest resource in an information society is not information
but attention. Klingberg’s book, interesting as it is, does not leave the
reader with any way around that. In any case, a great deal of the
“information” (such as my Ph.D. offer this morning) turns out to be noise,
rather than anything meaningful. It’s necessary to pay just enough attention
to decide not to pay any more attention – a kind of catch-22.
Which is why it sometimes feels like one’s brain is
being nibbled by carnivorous gnats. It would be good if Dr. Klingberg and
his colleagues would apply themselves to finding a salve. Or better yet, a
repellent.
"Better Learning With Sites and Sound," by Andy Guess, Inside
Higher Ed, December 3, 2008 ---
http://www.insidehighered.com/news/2008/12/03/audio
Students in four graduate courses at West Virginia
University worked on and submitted group projects in two different ways,
alternating for each assignment: using Microsoft Word to save, track
changes, add comments and send files back and forth as e-mail attachments;
and sharing files and editing them online using Buzzword. According to the
study, the students “were more likely to use graphics, charts, links, etc.
in Buzzword because of the ease of inclusion” than in Word, possibly as a
function of the interface’s comparative ease of use.
Perhaps more significantly, the study found that
they were “more likely to explain more complex concepts using a combination
of text and non-text based materials. The majority of participants ...
expressed the view that it was easier to express themselves at a higher
cognitive level when they could present material using multiple media
sources.” They also had higher levels of satisfaction.
Although the study had a small sample size, Ice
suggested in an interview that the “multiple forms of sensory input” such as
charts, links and graphics not only make the information more understandable
to the reader “but apparently ... students are learning more from that
process as well"; a process that’s not too different from the wiki editing
experience. He is preparing a larger follow-up study with at least six
different institutions around the world.
In theory, then, collaborations using Web-based
editing tools can potentially boost understanding, at least visually.
But learning doesn’t just occur in the visual
realm. Ice co-authored a study, currently under review, that examines how
listening to spoken words while also reading at the same time can improve
students’ learning experiences. In particular, he and his colleagues
attempted a method in which professors record comments on students’ written
assignments, which students can then listen to as they read along at
corresponding points in the text. They can also record their own responses
and continue back and forth in a sort of audio conversation.
While the Web-based collaboration tools are free,
Ice’s method makes use of embedded audio features in Adobe Acrobat Pro. If
institutions own the software, however, students can listen to the audio
(and record their own additions) on the free and commonly used Acrobat
Reader. (Adobe provided 60 copies of Acrobat Pro for the study but no
additional funding or support.)
The forthcoming paper found that students in the
audio study were at least three times more likely to take professors’
comments into account in their final assignments if they were in audio form
as opposed to written. What they found, Ice said, was that “students are
actually listening to the instructor and reading what they wrote so they
have two sensory modes working at the same time,” which could actually
improve cognition.
Since the paper was produced, Ice added, additional
research has confirmed that the findings are generalizable over many
different contexts, such as types of learners and types of institutions.
But a central component of the effect is what the
authors call the “asynchronous audio feedback” aspect of the comments: that
students can listen to previously recorded audio while they’re reading what
it is referring to.
“I’ve tried other methods, too, where you send the
students a document and then also send them a [separate] sound file, and the
effect is not nearly as strong; as a matter of fact, it’s barely significant
when you do that,” Ice said.
Continued in article
"15 Years of Cutting-Edge Thinking on
Understanding the Mind," Edited by Maria Popova, The Atlantic,
September 14, 2011 ---
http://www.theatlantic.com/life/archive/2011/09/15-years-of-cutting-edge-thinking-on-understanding-the-mind/245006/
Bringing Jonathan Haidt, Martin Seligman, Alison Gopnik, Steven Pinker,
Philip Zimbardo, and others together between two covers
For the past 15 years,
literary-agent-turned-crusader-of-human-progress John Brockman has been a
remarkable curator of curiosity, long before either "curator" or "curiosity"
was a frivolously tossed around buzzword. His
Edge.org
has become an epicenter of bleeding-edge insight
across science, technology and beyond, hosting conversations with some of
our era's greatest thinkers (and, once a year,
asking them some big questions). Last month marked
the release of
The Mind, the first volume in The Best of
Edge Series, presenting eighteen provocative, landmark pieces --
essays, interviews, transcribed talks -- from the Edge archive. The
anthology reads like a who's who of Brain Pickings favorites across
psychology, evolutionary biology, social science, technology, and more. And,
perhaps equally interestingly, the tome -- most of the materials in which
are available for free online -- is an implicit manifesto for the enduring
power of books as curatorial capsules of ideas. Brockman writes in the
book's introduction:
While there's no doubt
about the value of online presentations, the role of books, whether
bound and printed or presented electronically, is still an invaluable
way to present important ideas. Thus, we are pleased to be able to offer
this series of books to the public.
Here's a small
sampling of the treasure chest between
The Mind's covers:
In "Eudaemonia: The
Good Life" (2004), Martin Seligman, father of positive psychology whom you
might recall as the author of
Flourish and
Learned Optimism, one of our
7 essential books on optimism, explores what he
calls the "third form of happiness," which lies in:
...knowing what your
highest straights are and deploying those in the service of something
you believe in is larger than you are. There's no shortcut to that.
That's what life is about. There will likely be a pharmacology of
pleasure, and there may be a pharmacology of positive emotion generally,
but it's unlikely there'll be an interesting pharmacology of flow. And
it's impossible that there'll be a pharmacology of meaning.
In "Moral Psychology
and the Misunderstanding of Religion" (2007), psychologist Jonathan Haidt
(whose
The Happiness Hypothesis you might recall as
one of our
7 favorite books on happiness) notes:
[I]t might seem obvious to
you that contractual societies are good, modern, creative, and free,
whereas beehive societies reek of feudalism, fascism, and patriarchy.
And, as a secular liberal I agree that contractual societies such as
those of Western Europe offer the best hope for living peacefully
together in our increasingly diverse modern nations (although it remains
to be seen if Europe can solve its current diversity problems). I just
want to make one point, however, that should give constructualists
pause: surveys have long shown that religious believers in the United
States are happier, healthier, longer-lived, and more generous to
charity and to each other than are secular people.
In
"Amazing Babies" (2009), psychologist and philosopher Alison
Gopnik laid the foundations for her
The Philosophical Baby, one
of this year's
must-read books by TED Global speakers:
We've known
for a long time that human children are the best
learning machines in the universe, but it has always
been like the mystery of the hummingbirds. We know that
they fly, but we don't know how they can possibly do it.
We could say that babies learn, but we didn't know how.
Harvard's Steven Pinker, whose illuminating insights on
violence and human nature you
might recall and who penned one of our
5 favorite books on language,
wrote in "Organs of Computation" (1997), long before the
hype of contemporary quasi-sciences like neuromarketing:
Most of the
assumptions about the mind that underlie current
discussions are many decades out of date. [L]ook at the
commentaries on human affairs by pundits and social
critics. They say we're 'conditioned' to do this, or
'brainwashed' to do that, or 'socialized' to believe
such and such. Where do these ideas come from? From the
behaviorism of the 1920s, from bad Cold War movies from
the 1950s, from folklore about the effects of family
upbringing that behavior genetics has shown to be false.
The basic understanding that the human mind is a
remarkably complex processor of information, an 'organ
of extreme perfection and complication,' to use Darwin's
phrase, has not made it into the mainstream of
intellectual life.
Continued in article
Bob Jensen's threads on metacognition are at
http://faculty.trinity.edu/rjensen/265wp.htm
Appendix 8
The Importance of Paying Attention for Longer Periods of
Time
Once Mischel began analyzing the results, he noticed
that low delayers, the children who rang the bell quickly, seemed more likely to
have behavioral problems, both in school and at home. They got lower S.A.T.
scores. They struggled in stressful situations, often had trouble paying
attention, and found it difficult to maintain friendships. The child who could
wait fifteen minutes had an S.A.T. score that was, on average, two hundred and
ten points higher than that of the kid who could wait only thirty seconds.
"Don’t! The secret of self-control," by Jonah Lehrer, The New Yorker, May
18, 2009 ---
http://www.newyorker.com/reporting/2009/05/18/090518fa_fact_lehrer
Jim Mahar clued me into this link.
In the late nineteen-sixties, Carolyn Weisz, a
four-year-old with long brown hair, was invited into a “game room” at the
Bing Nursery School, on the campus of Stanford University. The room was
little more than a large closet, containing a desk and a chair. Carolyn was
asked to sit down in the chair and pick a treat from a tray of marshmallows,
cookies, and pretzel sticks. Carolyn chose the marshmallow. Although she’s
now forty-four, Carolyn still has a weakness for those air-puffed balls of
corn syrup and gelatine. “I know I shouldn’t like them,” she says. “But
they’re just so delicious!” A researcher then made Carolyn an offer: she
could either eat one marshmallow right away or, if she was willing to wait
while he stepped out for a few minutes, she could have two marshmallows when
he returned. He said that if she rang a bell on the desk while he was away
he would come running back, and she could eat one marshmallow but would
forfeit the second. Then he left the room.
Although Carolyn has no direct memory of the
experiment, and the scientists would not release any information about the
subjects, she strongly suspects that she was able to delay gratification.
“I’ve always been really good at waiting,” Carolyn told me. “If you give me
a challenge or a task, then I’m going to find a way to do it, even if it
means not eating my favorite food.” Her mother, Karen Sortino, is still more
certain: “Even as a young kid, Carolyn was very patient. I’m sure she would
have waited.” But her brother Craig, who also took part in the experiment,
displayed less fortitude. Craig, a year older than Carolyn, still remembers
the torment of trying to wait. “At a certain point, it must have occurred to
me that I was all by myself,” he recalls. “And so I just started taking all
the candy.” According to Craig, he was also tested with little plastic
toys—he could have a second one if he held out—and he broke into the desk,
where he figured there would be additional toys. “I took everything I
could,” he says. “I cleaned them out. After that, I noticed the teachers
encouraged me to not go into the experiment room anymore.”
Footage of these experiments, which were conducted
over several years, is poignant, as the kids struggle to delay gratification
for just a little bit longer. Some cover their eyes with their hands or turn
around so that they can’t see the tray. Others start kicking the desk, or
tug on their pigtails, or stroke the marshmallow as if it were a tiny
stuffed animal. One child, a boy with neatly parted hair, looks carefully
around the room to make sure that nobody can see him. Then he picks up an
Oreo, delicately twists it apart, and licks off the white cream filling
before returning the cookie to the tray, a satisfied look on his face.
Most of the children were like Craig. They
struggled to resist the treat and held out for an average of less than three
minutes. “A few kids ate the marshmallow right away,” Walter Mischel, the
Stanford professor of psychology in charge of the experiment, remembers.
“They didn’t even bother ringing the bell. Other kids would stare directly
at the marshmallow and then ring the bell thirty seconds later.” About
thirty per cent of the children, however, were like Carolyn. They
successfully delayed gratification until the researcher returned, some
fifteen minutes later. These kids wrestled with temptation but found a way
to resist.
The initial goal of the experiment was to identify
the mental processes that allowed some people to delay gratification while
others simply surrendered. After publishing a few papers on the Bing studies
in the early seventies, Mischel moved on to other areas of personality
research. “There are only so many things you can do with kids trying not to
eat marshmallows.”
But occasionally Mischel would ask his three
daughters, all of whom attended the Bing, about their friends from nursery
school. “It was really just idle dinnertime conversation,” he says. “I’d ask
them, ‘How’s Jane? How’s Eric? How are they doing in school?’ ” Mischel
began to notice a link between the children’s academic performance as
teen-agers and their ability to wait for the second marshmallow. He asked
his daughters to assess their friends academically on a scale of zero to
five. Comparing these ratings with the original data set, he saw a
correlation. “That’s when I realized I had to do this seriously,” he says.
Starting in 1981, Mischel sent out a questionnaire to all the reachable
parents, teachers, and academic advisers of the six hundred and fifty-three
subjects who had participated in the marshmallow task, who were by then in
high school. He asked about every trait he could think of, from their
capacity to plan and think ahead to their ability to “cope well with
problems” and get along with their peers. He also requested their S.A.T.
scores.
Once Mischel began analyzing the results, he
noticed that low delayers, the children who rang the bell quickly, seemed
more likely to have behavioral problems, both in school and at home. They
got lower S.A.T. scores. They struggled in stressful situations, often had
trouble paying attention, and found it difficult to maintain friendships.
The child who could wait fifteen minutes had an S.A.T. score that was, on
average, two hundred and ten points higher than that of the kid who could
wait only thirty seconds.
Carolyn Weisz is a textbook example of a high
delayer. She attended Stanford as an undergraduate, and got her Ph.D. in
social psychology at Princeton. She’s now an associate psychology professor
at the University of Puget Sound. Craig, meanwhile, moved to Los Angeles and
has spent his career doing “all kinds of things” in the entertainment
industry, mostly in production. He’s currently helping to write and produce
a film. “Sure, I wish I had been a more patient person,” Craig says.
“Looking back, there are definitely moments when it would have helped me
make better career choices and stuff.”
Mischel and his colleagues continued to track the
subjects into their late thirties—Ozlem Ayduk, an assistant professor of
psychology at the University of California at Berkeley, found that
low-delaying adults have a significantly higher body-mass index and are more
likely to have had problems with drugs—but it was frustrating to have to
rely on self-reports. “There’s often a gap between what people are willing
to tell you and how they behave in the real world,” he explains. And so,
last year, Mischel, who is now a professor at Columbia, and a team of
collaborators began asking the original Bing subjects to travel to Stanford
for a few days of experiments in an fMRI machine. Carolyn says she will be
participating in the scanning experiments later this summer; Craig completed
a survey several years ago, but has yet to be invited to Palo Alto. The
scientists are hoping to identify the particular brain regions that allow
some people to delay gratification and control their temper. They’re also
conducting a variety of genetic tests, as they search for the hereditary
characteristics that influence the ability to wait for a second marshmallow.
If Mischel and his team succeed, they will have
outlined the neural circuitry of self-control. For decades, psychologists
have focussed on raw intelligence as the most important variable when it
comes to predicting success in life. Mischel argues that intelligence is
largely at the mercy of self-control: even the smartest kids still need to
do their homework. “What we’re really measuring with the marshmallows isn’t
will power or self-control,” Mischel says. “It’s much more important than
that. This task forces kids to find a way to make the situation work for
them. They want the second marshmallow, but how can they get it? We can’t
control the world, but we can control how we think about it.”
Walter Mischel is a slight, elegant man with a
shaved head and a face of deep creases. He talks with a Brooklyn bluster and
he tends to act out his sentences, so that when he describes the marshmallow
task he takes on the body language of an impatient four-year-old. “If you
want to know why some kids can wait and others can’t, then you’ve got to
think like they think,” Mischel says.
Mischel was born in Vienna, in 1930. His father was
a modestly successful businessman with a fondness for café society and
Esperanto, while his mother spent many of her days lying on the couch with
an ice pack on her forehead, trying to soothe her frail nerves. The family
considered itself fully assimilated, but after the Nazi annexation of
Austria, in 1938, Mischel remembers being taunted in school by the Hitler
Youth and watching as his father, hobbled by childhood polio, was forced to
limp through the streets in his pajamas. A few weeks after the takeover,
while the family was burning evidence of their Jewish ancestry in the
fireplace, Walter found a long-forgotten certificate of U.S. citizenship
issued to his maternal grandfather decades earlier, thus saving his family.
The family settled in Brooklyn, where Mischel’s
parents opened up a five-and-dime. Mischel attended New York University,
studying poetry under Delmore Schwartz and Allen Tate, and taking studio-art
classes with Philip Guston. He also became fascinated by psychoanalysis and
new measures of personality, such as the Rorschach test. “At the time, it
seemed like a mental X-ray machine,” he says. “You could solve a person by
showing them a picture.” Although he was pressured to join his uncle’s
umbrella business, he ended up pursuing a Ph.D. in clinical psychology at
Ohio State.
But Mischel noticed that academic theories had
limited application, and he was struck by the futility of most personality
science. He still flinches at the naïveté of graduate students who based
their diagnoses on a battery of meaningless tests. In 1955, Mischel was
offered an opportunity to study the “spirit possession” ceremonies of the
Orisha faith in Trinidad, and he leapt at the chance. Although his research
was supposed to involve the use of Rorschach tests to explore the
connections between the unconscious and the behavior of people when
possessed, Mischel soon grew interested in a different project. He lived in
a part of the island that was evenly split between people of East Indian and
of African descent; he noticed that each group defined the other in broad
stereotypes. “The East Indians would describe the Africans as impulsive
hedonists, who were always living for the moment and never thought about the
future,” he says. “The Africans, meanwhile, would say that the East Indians
didn’t know how to live and would stuff money in their mattress and never
enjoy themselves.”
Mischel took young children from both ethnic groups
and offered them a simple choice: they could have a miniature chocolate bar
right away or, if they waited a few days, they could get a much bigger
chocolate bar. Mischel’s results failed to justify the stereotypes—other
variables, such as whether or not the children lived with their father,
turned out to be much more important—but they did get him interested in the
question of delayed gratification. Why did some children wait and not
others? What made waiting possible? Unlike the broad traits supposedly
assessed by personality tests, self-control struck Mischel as potentially
measurable.
Continued in article
Jensen Comment
I think this entire article has a lot to do with metacognition and the
importance of self-learning because it is often easier to pay attention when
learning on your own.
Bob Jensen's threads on learning memory are at
http://faculty.trinity.edu/rjensen/265wp.htm
How to Train the Aging Brain
"How to Train the Aging Brain," by Barbara Strauch, The New York
Times, December 29, 2009 ---
http://www.nytimes.com/2010/01/03/education/edlife/03adult-t.html?hpw
I LOVE reading history, and the shelves in my
living room are lined with fat, fact-filled books. There’s “The Hemingses of
Monticello,” about the family of Thomas Jefferson’s slave mistress; there’s
“House of Cards,” about the fall of Bear Stearns; there’s “Titan,” about
John D. Rockefeller Sr.
The problem is, as much as I’ve enjoyed these
books, I don’t really remember reading any of them. Certainly I know the
main points. But didn’t I, after underlining all those interesting parts,
retain anything else? It’s maddening and, sorry to say, not all that unusual
for a brain at middle age: I don’t just forget whole books, but movies I
just saw, breakfasts I just ate, and the names, oh, the names are awful. Who
are you?
Brains in middle age, which, with increased life
spans, now stretches from the 40s to late 60s, also get more easily
distracted. Start boiling water for pasta, go answer the doorbell and —
whoosh — all thoughts of boiling water disappear. Indeed, aging brains, even
in the middle years, fall into what’s called the default mode, during which
the mind wanders off and begin daydreaming.
Given all this, the question arises, can an old
brain learn, and then remember what it learns? Put another way, is this a
brain that should be in school?
As it happens, yes. While it’s tempting to focus on
the flaws in older brains, that inducement overlooks how capable they’ve
become. Over the past several years, scientists have looked deeper into how
brains age and confirmed that they continue to develop through and beyond
middle age.
Many longheld views, including the one that 40
percent of brain cells are lost, have been overturned. What is stuffed into
your head may not have vanished but has simply been squirreled away in the
folds of your neurons.
One explanation for how this occurs comes from
Deborah M. Burke, a professor of psychology at Pomona College in California.
Dr. Burke has done research on “tots,” those tip-of-the-tongue times when
you know something but can’t quite call it to mind. Dr. Burke’s research
shows that such incidents increase in part because neural connections, which
receive, process and transmit information, can weaken with disuse or age.
But she also finds that if you are primed with
sounds that are close to those you’re trying to remember — say someone talks
about cherry pits as you try to recall Brad Pitt’s name — suddenly the lost
name will pop into mind. The similarity in sounds can jump-start a limp
brain connection. (It also sometimes works to silently run through the
alphabet until landing on the first letter of the wayward word.)
This association often happens automatically, and
goes unnoticed. Not long ago I started reading “The Prize,” a history of the
oil business. When I got to the part about Rockefeller’s early days as an
oil refinery owner, I realized, hey, I already know this from having read
“Titan.” The material was still in my head; it just needed a little prodding
to emerge.
Recently, researchers have found even more positive
news. The brain, as it traverses middle age, gets better at recognizing the
central idea, the big picture. If kept in good shape, the brain can continue
to build pathways that help its owner recognize patterns and, as a
consequence, see significance and even solutions much faster than a young
person can.
The trick is finding ways to keep brain connections
in good condition and to grow more of them.
“The brain is plastic and continues to change, not
in getting bigger but allowing for greater complexity and deeper
understanding,” says Kathleen Taylor, a professor at St. Mary’s College of
California, who has studied ways to teach adults effectively. “As adults we
may not always learn quite as fast, but we are set up for this next
developmental step.”
Educators say that, for adults, one way to nudge
neurons in the right direction is to challenge the very assumptions they
have worked so hard to accumulate while young. With a brain already full of
well-connected pathways, adult learners should “jiggle their synapses a bit”
by confronting thoughts that are contrary to their own, says Dr. Taylor, who
is 66.
Teaching new facts should not be the focus of adult
education, she says. Instead, continued brain development and a richer form
of learning may require that you “bump up against people and ideas” that are
different. In a history class, that might mean reading multiple viewpoints,
and then prying open brain networks by reflecting on how what was learned
has changed your view of the world.
“There’s a place for information,” Dr. Taylor says.
“We need to know stuff. But we need to move beyond that and challenge our
perception of the world. If you always hang around with those you agree with
and read things that agree with what you already know, you’re not going to
wrestle with your established brain connections.”
Such stretching is exactly what scientists say best
keeps a brain in tune: get out of the comfort zone to push and nourish your
brain. Do anything from learning a foreign language to taking a different
route to work.
“As adults we have these well-trodden paths in our
synapses,” Dr. Taylor says. “We have to crack the cognitive egg and scramble
it up. And if you learn something this way, when you think of it again
you’ll have an overlay of complexity you didn’t have before — and help your
brain keep developing as well.”
Jack Mezirow, a professor emeritus at Columbia
Teachers College, has proposed that adults learn best if presented with what
he calls a “disorienting dilemma,” or something that “helps you critically
reflect on the assumptions you’ve acquired.”
Dr. Mezirow developed this concept 30 years ago
after he studied women who had gone back to school. The women took this bold
step only after having many conversations that helped them “challenge their
own ingrained perceptions of that time when women could not do what men
could do.”
Such new discovery, Dr. Mezirow says, is the
“essential thing in adult learning.”
“As adults we have all those brain pathways built
up, and we need to look at our insights critically,” he says. “This is the
best way for adults to learn. And if we do it, we can remain sharp.”
And so I wonder, was my cognitive egg scrambled by
reading that book on Thomas Jefferson? Did I, by exploring the flaws in a
man I admire, create a suitably disorienting dilemma? Have I, as a result,
shaken up and fed a brain cell or two?
And perhaps it doesn’t matter that I can’t, at
times, recall the given name of the slave with whom Jefferson had all those
children. After all, I can Google a simple name.
Sally.
Barbara Strauch is The Times’s health editor; her book “The Secret
Life of the Grown-Up Brain” will be published in April.
Jensen Comment
At my age, this NYT article hits close to home. One way I exercise my aging
brain is messaging to the AECM.
One thing I most certainly note in my old brain are “tots” --- those
tip-of-the-tongue times. What helps my memory is a massive Web site that lets me
look up tidbits that jog my memory --- I should call it my Website for Tots ---
http://faculty.trinity.edu/rjensen/threads.htm
Appendix 10
That Placebo Effect in Research: Dan Ariely on Tennis Shoes
and Toilet Paper
Dan Ariely is James B. Duke Professor of Behavioral Economics at Duke University
and is head of the eRationality research group at the
MIT Media Lab.
Dan Ariely ---
http://en.wikipedia.org/wiki/Dan_Ariely
Published works
-
Ariely, Dan (2008).
Predictably Irrational: The Hidden Forces That Shape Our Decisions.
HarperCollins. pp. 304.
ISBN
9780061353239.
-
Try it, you'll like it: The influence of expectation, consumption, and
revelation on preferences for beer
-
Dishonesty in Everyday Life and Its Policy Implications
-
Placebo Effects of Marketing Actions: Consumers May Get What They Pay
For
-
Tom Sawyer and the Construction of Value
- Heyman,
James; Ariely, Dan (2004). "Effort
for Payment: A Tale of Two markets". Psychological Science
15 (11): 787-793(7).
http://web.mit.edu/ariely/www/MIT/Papers/2markets.pdf.
-
Ariely, Dan; Wertenbroch, Klaus
(2002). "Procrastination,
Deadlines, and Performance: Self-Control by Precommitment".
Psychological Science 13 (3): 219-224.
http://web.mit.edu/ariely/www/MIT/Papers/deadlines.pdf.
-
Ariely, Dan (2001). "Seeing
sets: Representation by statistical properties". Psychological
Science 12 (2): 157-162.
http://web.mit.edu/ariely/www/MIT/Papers/spot.pdf.
-
Ariely, Dan (2000). "Controlling
information flow: Effects on consumers' decision making and preference".
Journal of Consumer Research 27 (2): 233-248.
doi:10.1086/314322.
http://www.journals.uchicago.edu/cgi-bin/resolve?id=doi:10.1086/314322.
-
Coherent Arbitrariness: Stable demand curves without stable preferences
-
Combining experiences over time: the effects of duration, intensity
changes and on-line measurements on retrospective pain evaluations
-
Ariely, Dan; Zauberman, Gal (2000). "On
the making of an experience: The effects of breaking and combining
experiences on their overall evaluation". Journal of Behavioral
Decision Making 13: 219-232.
http://web.mit.edu/ariely/www/MIT/Papers/expseg.pdf.
"The Science Behind Exercise Footwear," by Dan Ariely, MIT's
Technology Review, January 5, 2010 ---
http://www.technologyreview.com/blog/post.aspx?bid=355&bpid=24614&nlid=2647
A few weeks ago Reebok unveiled a walking shoe
purported to tone muscles to a greater extent than your average sneaker. All
you had to do was slip on a pair of EasyTone and the rest would take care of
itself.
Exercise without exercise? Great!
Considering the abracadabra-like quality of the
shoe, it’s no surprise that it’s been selling like hotcakes. The question of
course is “ does it work”?
According to a
recent New York Times article on the topic Reebok
has accumulated “15,000 hours’ worth of wear-test data from shoe users who
say they notice the difference.” (The company also quotes a study as
support, but it’s one they commissioned themselves and only carries a sample
size of five.) The two women quoted in the article further echo this
sentiment.
Reebok’s head of advanced innovation (and EasyTone
mastermind), Bill McInnis, says the shoe works because it offers the kind of
imbalance that you get with stability balls at the gym. Unlike other
sneakers, which are made flat with comfort in mind, the EasyTone is
purposely outfitted with air-filled toe-and-heal “balance pods” in order to
simulate the muscle engagement required to walk through sand. With every
step, air shifts from one pod to the other, causing the person’s foot to
sink and forcing their leg and backside muscles into a workout.
But as the Times article proposes at the end
(without explicitly using the term), the shoe’s success could instead come
from the placebo effect. Thanks to Reebok’s marketing efforts, buyers pick
up the shoes already convinced of their success, a mind frame that may then
cause them to walk faster or harder or longer, thereby producing the
expected workout – just not for the expected reason.
And there are some reasons to suspect this kind of
placebo effect: In a paper by Alia Crum and Ellen Langer. Titled “Mind-Set
Matters: Exercise and the Placebo Effect.” In their research they told some
maids working in hotels that the work they do (cleaning hotel rooms) is good
exercise and satisfies the Surgeon General’s recommendations for an active
lifestyle. Other maids were not given this information. 4 weeks later, the
informed group perceived themselves to be getting significantly more
exercise than before, their weight was lower and they even showed a decrease
in blood pressure, body fat, waist-to-hip ratio, and body mass index.
So, maybe exercise affects health are part placebo?
Irrationally Yours
Dan
A One-Hour Video on What it Means to
Be Predictably Irrational (July 25, 2008) ---
http://financialrounds.blogspot.com/
The video is also at
http://www.youtube.com/watch?v=VZv--sm9XXU
This is quite interesting!
From the Financial Rounds Blog on
January 25, 2008 ---
http://financialrounds.blogspot.com/
"Dan Ariely (Duke University) -
Predictably Irrational
Here's a
video of Dan Ariely (author of "Predictably
Irrational") in his recent talk for the Google Authors program. Ariely
has written a fascinating book about some of the cognitive and behavioral
biases that most of us exhibit. If you listen carefully, you'll find that he
even gives a hint about how to increase your student evaluations ---
http://financialrounds.blogspot.com/
Summary of what it means to be
"predictably irrational" ---
http://en.wikipedia.org/wiki/Predictably_Irrational
New York Times Book Review
"Emonomics," by David Berreby, The New York Times, March 16, 2008
---
http://www.nytimes.com/2008/03/16/books/review/Berreby-t.html?_r=1&oref=slogin
For years, the
ideology of free markets bestrode the world, bending politics as well as
economics to its core assumption: market forces produce the best solution to
any problem. But these days, even Bill Gates says capitalism’s work is
“unsatisfactory” for one-third of humanity, and not even Hillary Clinton
supports Bill Clinton’s 1990s trade pacts.
Another sign that
times are changing is “Predictably Irrational,” a book that both exemplifies
and explains this shift in the cultural winds. Here, Dan Ariely, an
economist at M.I.T., tells us that “life with fewer market norms and more
social norms would be more satisfying, creative, fulfilling and fun.” By the
way, the conference where he had this insight wasn’t sponsored by the
Federal Reserve, where he is a researcher. It came to him at Burning Man,
the annual anarchist conclave where clothes are optional and money is
banned. Ariely calls it “the most accepting, social and caring place I had
ever been.”
Obviously, this sly
and lucid book is not about your grandfather’s dismal science. Ariely’s
trade is behavioral economics, which is the study, by experiments, of what
people actually do when they buy, sell, change jobs, marry and make other
real-life decisions.
To see how arousal
alters sexual attitudes, for example, Ariely and his colleagues asked young
men to answer a questionnaire — then asked them to answer it again, only
this time while indulging in Internet pornography on a laptop wrapped in
Saran Wrap. (In that state, their answers to questions about sexual tastes,,
violence and condom use were far less respectable.) To study the power of
suggestion, Ariely’s team zapped volunteers with a little painful
electricity, then offered fake pain pills costing either 10 cents or $2.50
(all reduced the pain, but the more expensive ones had a far greater
effect). To see how social situations affect honesty, they created tests
that made it easy to cheat, then looked at what happened if they reminded
people right before the test of a moral rule. (It turned out that being
reminded of any moral code — the Ten Commandments, the non-existent “M.I.T.
honor system” — caused cheating to plummet.)
These sorts of
rigorous but goofy-sounding experiments lend themselves to a genial,
gee-whiz style, with which Ariely moves comfortably from the lab to broad
social questions to his own life (why did he buy that Audi instead of a
sensible minivan?). He is good-tempered company — if he mentions you in this
book, you are going to be called “brilliant,” “fantastic” or “delightful” —
and crystal clear about all he describes. But “Predictably Irrational” is a
far more revolutionary book than its unthreatening manner lets on. It’s a
concise summary of why today’s social science increasingly treats the
markets-know-best model as a fairy tale.
At the heart of the
market approach to understanding people is a set of assumptions. First, you
are a coherent and unitary self. Second, you can be sure of what this self
of yours wants and needs, and can predict what it will do. Third, you get
some information about yourself from your body — objective facts about
hunger, thirst, pain and pleasure that help guide your decisions. Standard
economics, as Ariely writes, assumes that all of us, equipped with this sort
of self, “know all the pertinent information about our decisions” and “we
can calculate the value of the different options we face.” We are, for
important decisions, rational, and that’s what makes markets so effective at
finding value and allocating work. To borrow from H. L. Mencken, the market
approach presumes that “the common people know what they want, and deserve
to get it good and hard.”
What the past few
decades of work in psychology, sociology and economics has shown, as Ariely
describes, is that all three of these assumptions are false. Yes, you have a
rational self, but it’s not your only one, nor is it often in charge. A more
accurate picture is that there are a bunch of different versions of you, who
come to the fore under different conditions. We aren’t cool calculators of
self-interest who sometimes go crazy; we’re crazies who are, under special
circumstances, sometimes rational.
Ariely is not out
to overthrow rationality. Instead, he and his fellow social scientists want
to replace the “rational economic man” model with one that more accurately
describes the real laws that drive human choices. In a chapter on
“relativity,” for example, Ariely writes that evaluating two houses side by
side yields different results than evaluating three — A, B and a somewhat
less appealing version of A. The subpar A makes it easier to decide that A
is better — not only better than the similar one, but better than B. The
lesser version of A should have no effect on your rating of the other two
buildings, but it does. Similarly, he describes the “zero price effect,”
which marketers exploit to convince us to buy something we don’t really want
or need in order to collect a “free” gift. “FREE! gives us such an emotional
charge that we perceive what is being offered as immensely more valuable
than it really is,” Ariely writes. None of this is rational, but it is
predictable.
What the reasoning
self should do, he says, is set up guardrails to manage things during those
many, many moments when reason is not in charge. (Though one might ask why
the reasoning self should always be in charge, an assumption Ariely doesn’t
examine too closely.)
For example,
Ariely writes, we know our irrational self falls easily into wanting stuff
we can’t afford and don’t need. So he proposes a credit card that encourages
planning and self-control. After $50 is spent on chocolate this month — pfft,
declined! He has in fact suggested this to a major bank. Of course, he knew
that his idea would cut into the $17 billion a year that American banks make
on consumer credit-card interest, but what the heck: money isn’t everything.
An Experiment With Toilet Paper and
Other Messages ---
http://www.predictablyirrational.com/
Other videos on being Predictably
Irrational
Great Minds in Management: The
Process of Theory Development ---
http://faculty.trinity.edu/rjensen//theory/00overview/GreatMinds.htm
CBS Sixty Minutes Video
Mozart of Chess: Magness Carlsen
The Youngest Number One Chess Player in History
Magnus is number one in the chess world. That is
sensational. In a very short segment, there are any number of possible
"angles" to convey his remarkable talents and achievements to the public at
large. While admittedly hackneyed in virtually all respects, I generally
have no substantive quarrel with most of the editorial selections of 60
Minutes in this regard. But the part about the potential of following in the
footsteps of Bobby Fischer's mental illness seems deceptively truncated. I
find it very hard to believe that, in any extended discussion of the
subject, Magnus would really leave one with the impression that he is truly
concerned that he will end up in the same sorry state that Fischer did. The
problematic -- or symptomatic -- details of Fischer's early life (including
up until Magnus's current age, for example) are well known. In any
discussion, surely Magnus would convey that, while the Fischer saga
naturally may give rise to momentary self-reflective wonder, it is plain
that Magnus's own family life, overall socialization, behavior, and outlook
bear no resemblance whatsoever to that of Fischer. Anything Magnus conveyed
along these lines obviously ended up on the cutting room floor.
I applaud all the attention that Magnus garners
within the mainstream media. It's good for chess. His publicist would be
wise, however, to develop a list of questions and "angles" to explore that
are more interesting and penetrating (even for a general audience) than what
we see again and again.
Jensen Comment
This was one of the most interesting television segments I ever watched.
The video demonstrates how master chess players are super confident with a
compulsion not just to beat their opponents but to annihilate them.
The only opponent to ever intimidate Magness was the great Russian great
Garry Kasparov.
Kasparov was always one of my heroes until I learned how rude and
unsportsmanlike he was in a match with Magness Carlsen (a kid). This is probably
the only time in the world that Magness was intimidated to a point where he
settled for a tie when he could've won the match.
"How experts recall chess positions," by Daniel Simons, The
Invisible Gorilla, February 15, 2012 ---
http://theinvisiblegorilla.com/blog/2012/02/15/how-experts-recall-chess-positions/
. . .
This question, how do chess experts evaluate
positions to find the best move, has been studied for decades, dating back
to the groundbreaking work of
Adriaan de Groot
and later to work by William Chase and
Herbert Simon.
de Groot interviewed several chess players as they
evaluated positions, and he argued that experts and weaker players tended to
“look” about the same number of moves ahead and to evaluate similar numbers
of moves with roughly similar speed. The relatively small differences
between experts and novices suggested that their advantages came not from
brute force calculation ability but from something else: knowledge.
According to De Groot, the core of chess expertise is the ability to
recognize huge number of chess positions (or parts of positions) and to
derive moves from them. In short, their greater efficiency came not from
evaluating more outcomes, but from considering only the better options. [Note:
Some of the details of de Groot’s claims, which he made before the
appropriate statistical tests were in widespread use, did not hold up to
later scrutiny—experts do consider somewhat more options, look a bit deeper,
and process positions faster than less expert players (Holding, 1992). But
de Groot was right about the limited nature of expert search and the
importance of knowledge and pattern recognition in expert performance.]
Continued in the article (with an interesting concluding video)
Appendix 11
Computer Trained Yet Deeply Intuitive
"The Future of Decision Making: Less
Intuition, More Evidence," Simoleon Sense, January 11, 2010 ---
http://www.simoleonsense.com/the-future-of-decision-making-less-intuition-more-evidence/
Awesome article (covering decision making, Kahneman, etc) via Harvard
Big thanks & h/t to Michael & Stuart
Click Here Fore: The Future of Decision Making: Less Intuition, More
Evidence
Introduction (Via Harvard Blogs)
Human intuition can be astonishingly good,
especially after it’s improved by experience. Savvy poker players are so
good at reading their opponents’ cards and bluffs that they seem to have
x-ray vision. Firefighters can, under extreme duress, anticipate how
flames will spread through a building. And nurses in neonatal ICUs can
tell if a baby has a dangerous infection even before blood test results
come back from the lab.
The lexicon to describe this phenomenon is
mostly mystical in nature. Poker players have a sixth sense;
firefighters feel the blaze’s intentions; Nurses just know what seems
like an infection. They can’t even tell us what data and cues they use
to make their excellent judgments; their intuition springs from a deep
place that can’t be easily examined. . Examples like these give many
people the impression that human intuition is generally reliable, and
that we should rely more on the decisions and predictions that come to
us in the blink of an eye.
Findings (Via Harvard Blogs)
* It takes a long time to build good intuition. Chess
players, for example, need 10 years of dedicated study and competition
to assemble a sufficient mental repertoire of board patterns.
* Intuition only works well in specific environments, ones that
provide a person with good cues and rapid feedback . Cues are
accurate indications about what’s going to happen next. They exist in
poker and firefighting, but not in, say, stock markets. Despite what
chartists think, it’s impossible to build good intuition about future
market moves because no publicly available information provides good
cues about later stock movements. Feedback from the environment is
information about what worked and what didn’t. It exists in neonatal
ICUs because babies stay there for a while. It’s hard, though, to build
medical intuition about conditions that change after the patient has
left the care environment, since there’s no feedback loop.
* We apply intuition inconsistently. Even experts are
inconsistent. One study determined what criteria clinical psychologists
used to diagnose their patients, and then created simple models based on
these criteria. Then, the researchers presented the doctors with new
patients to diagnose and also diagnosed those new patients with their
models. The models did a better job diagnosing the new cases than did
the humans whose knowledge was used to build them. The best explanation
for this is that people applied what they knew inconsistently — their
intuition varied. Models, though, don’t have intuition.
* It’s easy to make bad judgments quickly. We have a
many biases that lead us astray when making assessments. Here’s just one
example. If I ask a group of people “Is the average price of German cars
more or less than $100,000?” and then ask them to estimate the average
price of German cars, they’ll “anchor” around BMWs and other high-end
makes when estimating. If I ask a parallel group the same two questions
but say “more or less than $30,000″ instead, they’ll anchor around VWs
and give a much lower estimate. How much lower? About $35,000 on
average, or half the difference in the two anchor prices. How
information is presented affects what we think.
* We can’t know tell where our ideas
come from. There’s no way for even an experienced person to
know if a spontaneous idea is the result of legitimate expert intuition
or of a pernicious bias. In other words, we have lousy intuition about
our intuition.
Click Here Fore: The Future of Decision Making: Less Intuition, More
Evidence
"Video: Daniel Kahneman - The Psychology of Large Mistakes and Important
Decisions" Simoleon Sense, July 27, 2009 ---
http://www.simoleonsense.com/daniel-kahneman-psychology-of-large-mistakes-and-decisions/
"I was going to say I saw a duckie and a horsie, but I changed my mind."
Computer Trained Yet Deeply Intuitive
Not that Carlsen lacks computational prowess,
though. He often calculates 20 moves ahead and can comfortably play several
games simultaneously while blindfolded simply by hearing each move in
notation. The fear surrounding any such beautiful mind is that a life spent
probing the edges of the infinite — the possible permutations of a chess
game outnumber the estimated number of atoms in the universe — will
eventually lead to madness. Grand masters say Carlsen's precociousness is
reminiscent of Bobby Fischer's. The great American player spent his later
years in isolation, reappearing only to spout anti-Semitic conspiracy
theories. "It's easy to get obsessed with chess," Carlsen says. "That's what
happened with Fischer and Paul Morphy," another prodigy lost to madness. "I
don't have that same obsession."
Remember this Charles Shultz Cartoon
- Lucy Van Pelt: Aren't the clouds
beautiful? They look like big balls of cotton. I could just lie here all
day and watch them drift by. If you use your imagination, you can see
lots of things in the cloud's formations. What do you think you see,
Linus?
-
- Linus Van Pelt: Well, those clouds up
there look to me look like the map of the
British Honduras on the Caribbean. [points
up] That cloud up there looks a little like the profile of
Thomas Eakins, the famous painter and
sculptor. And that group of clouds over there... [points]
...gives me the impression of the
Stoning of Stephen. I can see the
Apostle Paul standing there to one side.
-
- Lucy Van Pelt: Uh huh. That's very
good. What do you see in the clouds, Charlie Brown?
-
- Charlie Brown: Well... I was going to
say I saw a duckie and a horsie, but I changed my mind.
"A Bold Opening for Chess Player," by Magnus Carlsen, Time Magazine,
January 11, 2010, Page 43 ---
http://www.time.com/time/magazine/article/0,9171,1950683,00.html
Vladimir Kramnik, former world chess
champion and current No. 4, is playing in the first round of the London
Chess Classic, the most competitive chess tournament to be played in the
U.K. capital in 25 years. Tall, handsome and expressionless, he looks
exactly as a man who has mastered a game of nearly infinite variation
should: like a high-end assassin. Today, however, he is getting methodically
and mercilessly crushed.
His opponent is a teenager who seems to be
having difficulty staying awake. Magnus Carlsen yawns, fidgets, slumps in
his chair. He gets up and wanders over to the other games, staring at the
boards like a curious toddler. Every now and then, he returns to his own
game and moves one of his pieces, inexorably building an attack so fierce
that by the 43rd move Kramnik sees the hopelessness of his position and
resigns.
Genius can appear anywhere, but the
origins of Carlsen's talent are particularly mysterious. In November,
Carlsen, then 18, became the youngest world No. 1 in the game's history. He
hails from Norway — a "small, poxy chess nation with almost no history of
success," as the English grand master Nigel Short sniffily describes it —
and unlike many chess prodigies who are full-time players by age 12, Carlsen
stayed in school until last year. His father Henrik, a soft-spoken engineer,
says he has spent more time urging his young son to complete his schoolwork
than to play chess. Even now, Henrik will interrupt Carlsen's chess studies
to drag him out for a family hike or museum trip. "I still have to pinch my
arm," Henrik says. "This certainly is not what we had in mind for Magnus."
Even pro chess players — a population
inured to demonstrations of extraordinary intellect — have been electrified
by Carlsen's rise. A grand master at 13 (the third youngest in history) and
a conqueror of top players at 15, he is often referred to as the Mozart of
chess for the seeming ease of his mastery. In September, he announced a
coaching contract with Garry Kasparov, arguably the greatest player of all
time, who quit chess in 2005 to pursue a political career in Russia. "Before
he is done," Kasparov says, "Carlsen will have changed our ancient game
considerably."
In conversation, Carlsen offers only
subtle clues to his intelligence. His speech, like his chess, is technical,
grammatically flawless and logically irresistible. He dresses neatly but
shows a teenager's discomfort with formality. (He rarely makes it through a
game without his shirt coming untucked.) He would seem older than 19 but for
his habit of giggling and his coltlike aversion to eye contact.
Carlsen joins chess's élite at a time of
unprecedented change. He is one of a generation of players who learned the
game from computers. To this day, he's not certain if he has an actual board
at home. "I might have one somewhere. I'm not sure," he says. Powerful chess
programs, which now routinely beat the best human competitors, have allowed
grand masters to study positions at a deeper level than was possible before.
Short says top players can now spend almost an entire game trading moves
that have been scripted by the same program and that such play by rote has
removed some of the mystique of chess. He likens chess computers to
"chainsaws chopping down the Amazon." (Read a Q&A with Carlsen.)
But Kasparov says Carlsen's mastery is
rooted in a "deep intuitive sense no computer can teach" and that his pupil
"has a natural feel for where to place the pieces." According to Kasparov,
Carlsen has a knack for sensing the potential energy in each move, even if
its ultimate effect is too far away for anyone — even a computer — to
calculate. In the grand-master commentary room, where chess's clerisy gather
to analyze play, the experts did not even consider several of Carlsen's
moves during his game with Kramnik until they saw them and realized they
were perfect. "It's hard to explain," Carlsen says. "Sometimes a move just
feels right."
Not that Carlsen lacks computational prowess,
though. He often calculates 20 moves ahead and can comfortably play several
games simultaneously while blindfolded simply by hearing each move in
notation. The fear surrounding any such beautiful mind is that a life spent
probing the edges of the infinite — the possible permutations of a chess
game outnumber the estimated number of atoms in the universe — will
eventually lead to madness. Grand masters say Carlsen's precociousness is
reminiscent of Bobby Fischer's. The great American player spent his later
years in isolation, reappearing only to spout anti-Semitic conspiracy
theories. "It's easy to get obsessed with chess," Carlsen says. "That's what
happened with Fischer and Paul Morphy," another prodigy lost to madness. "I
don't have that same obsession." (Read: "Fischer
vs. Spassky: Battle of the Brains.")
Although firmly atop the chess rankings,
thanks in part to his victory in London, Carlsen must now fight his way
through a series of qualifying competitions in order to earn a chance to
play for the world-championship title — the game's highest prize, which is
contested every two or three years. His father says he is more concerned
about "whether chess will make him a happy person." It seems to be doing
just that. "I love the game. I love to compete," Carlsen says. Asked how
long he will continue to enjoy chess and where the game will take him,
Carlsen pauses to ponder the variables. "It's too difficult to predict," he
concludes. So far, at least, he's been making all the right moves.
Fascinating History
"The Chess Master and the Computer," By Garry Kasparov, New York
Books, February 11, 2010 ---
http://www.nybooks.com/articles/23592
In 1985, in Hamburg, I played against thirty-two
different chess computers at the same time in what is known as a
simultaneous exhibition. I walked from one machine to the next, making my
moves over a period of more than five hours. The four leading chess computer
manufacturers had sent their top models, including eight named after me from
the electronics firm Saitek.
It illustrates the state of computer chess at the
time that it didn't come as much of a surprise when I achieved a perfect
32–0 score, winning every game, although there was an uncomfortable moment.
At one point I realized that I was drifting into trouble in a game against
one of the "Kasparov" brand models. If this machine scored a win or even a
draw, people would be quick to say that I had thrown the game to get PR for
the company, so I had to intensify my efforts. Eventually I found a way to
trick the machine with a sacrifice it should have refused. From the human
perspective, or at least from my perspective, those were the good old days
of man vs. machine chess.
Eleven years later I narrowly defeated the
supercomputer Deep Blue in a match. Then, in 1997, IBM redoubled its
efforts—and doubled Deep Blue's processing power—and I lost the rematch in
an event that made headlines around the world. The result was met with
astonishment and grief by those who took it as a symbol of mankind's
submission before the almighty computer. ("The Brain's Last Stand" read the
Newsweek headline.) Others shrugged their shoulders, surprised that
humans could still compete at all against the enormous calculating power
that, by 1997, sat on just about every desk in the first world.
It was the specialists—the chess players and the
programmers and the artificial intelligence enthusiasts—who had a more
nuanced appreciation of the result. Grandmasters had already begun to see
the implications of the existence of machines that could play—if only, at
this point, in a select few types of board configurations—with godlike
perfection. The computer chess people were delighted with the conquest of
one of the earliest and holiest grails of computer science, in many cases
matching the mainstream media's hyperbole. The 2003 book Deep Blue by
Monty Newborn was blurbed as follows: "a rare, pivotal watershed beyond all
other triumphs: Orville Wright's first flight, NASA's landing on the
moon...."
The AI crowd, too, was pleased with the result and
the attention, but dismayed by the fact that Deep Blue was hardly what their
predecessors had imagined decades earlier when they dreamed of creating a
machine to defeat the world chess champion. Instead of a computer that
thought and played chess like a human, with human creativity and intuition,
they got one that played like a machine, systematically evaluating 200
million possible moves on the chess board per second and winning with brute
number-crunching force. As Igor Aleksander, a British AI and neural networks
pioneer, explained in his 2000 book, How to Build a Mind:
By the mid-1990s the number of people with some
experience of using computers was many orders of magnitude greater than
in the 1960s. In the Kasparov defeat they recognized that here was a
great triumph for programmers, but not one that may compete with the
human intelligence that helps us to lead our lives.
It was an impressive achievement, of course, and a
human achievement by the members of the IBM team, but Deep Blue was
only intelligent the way your programmable alarm clock is intelligent. Not
that losing to a $10 million alarm clock made me feel any better.
My hopes for a return match with
Deep Blue were dashed, unfortunately. IBM had the publicity it wanted and
quickly shut down the project. Other chess computing projects around the
world also lost their sponsorship. Though I would have liked my chances in a
rematch in 1998 if I were better prepared, it was clear then that computer
superiority over humans in chess had always been just a matter of time.
Today, for $50 you can buy a home PC program that will crush most
grandmasters. In 2003, I played serious matches against two of these
programs running on commercially available multiprocessor servers—and, of
course, I was playing just one game at a time—and in both cases the score
ended in a tie with a win apiece and several draws.
Inevitable or not, no one understood all the
ramifications of having a super-grandmaster on your laptop, especially what
this would mean for professional chess. There were many doomsday scenarios
about people losing interest in chess with the rise of the machines,
especially after my loss to Deep Blue. Some replied to this with variations
on the theme of how we still hold footraces despite cars and bicycles going
much faster, a spurious analogy since cars do not help humans run faster
while chess computers undoubtedly have an effect on the quality of human
chess.
Another group postulated that the game would be
solved, i.e., a mathematically conclusive way for a computer to win from the
start would be found. (Or perhaps it would prove that a game of chess played
in the best possible way always ends in a draw.) Perhaps a real version of
HAL 9000 would simply announce move 1.e4, with checkmate in, say, 38,484
moves. These gloomy predictions have not come true, nor will they ever come
to pass. Chess is far too complex to be definitively solved with any
technology we can conceive of today. However, our looked-down-upon cousin,
checkers, or draughts, suffered this fate quite recently thanks to the work
of Jonathan Schaeffer at the University of Alberta and his unbeatable
program Chinook.
The number of legal chess positions is 1040,
the number of different possible games, 10120. Authors have
attempted various ways to convey this immensity, usually based on one of the
few fields to regularly employ such exponents, astronomy. In his book
Chess Metaphors, Diego Rasskin-Gutman points out that a player looking
eight moves ahead is already presented with as many possible games as there
are stars in the galaxy. Another staple, a variation of which is also used
by Rasskin-Gutman, is to say there are more possible chess games than the
number of atoms in the universe. All of these comparisons impress upon the
casual observer why brute-force computer calculation can't solve this
ancient board game. They are also handy, and I am not above doing this
myself, for impressing people with how complicated chess is, if only in a
largely irrelevant mathematical way.
This astronomical scale is not at all irrelevant to
chess programmers. They've known from the beginning that solving the
game—creating a provably unbeatable program—was not possible with the
computer power available, and that effective shortcuts would have to be
found. In fact, the first chess program put into practice was designed by
legendary British mathematician Alan Turing in 1952, and he didn't even have
a computer! He processed the algorithm on pieces of paper and this "paper
machine" played a competent game.
Rasskin-Gutman covers this well-traveled territory
in a book that achieves its goal of being an overview of overviews, if
little else. The history of the study of brain function is covered in the
first chapter, tempting the reader to skip ahead. You might recall axons and
dendrites from high school biology class. We also learn about cholinergic
and aminergic systems and many other things that are not found by my
computer's artificially intelligent English spell-checking system—or
referenced again by the author. Then it's on to similarly concise, if
inconclusive, surveys of artificial intelligence, chess computers, and how
humans play chess.
There have been many unintended
consequences, both positive and negative, of the rapid proliferation of
powerful chess software. Kids love computers and take to them naturally, so
it's no surprise that the same is true of the combination of chess and
computers. With the introduction of super-powerful software it became
possible for a youngster to have a top- level opponent at home instead of
need ing a professional trainer from an early age. Countries with little by
way of chess tradition and few available coaches can now produce prodigies.
I am in fact coaching one of them this year, nineteen-year-old Magnus
Carlsen, from Norway, where relatively little chess is played.
The heavy use of computer analysis has pushed the
game itself in new directions. The machine doesn't care about style or
patterns or hundreds of years of established theory. It counts up the values
of the chess pieces, analyzes a few billion moves, and counts them up again.
(A computer translates each piece and each positional factor into a value in
order to reduce the game to numbers it can crunch.) It is entirely free of
prejudice and doctrine and this has contributed to the development of
players who are almost as free of dogma as the machines with which they
train. Increasingly, a move isn't good or bad because it looks that way or
because it hasn't been done that way before. It's simply good if it works
and bad if it doesn't. Although we still require a strong measure of
intuition and logic to play well, humans today are starting to play more
like computers.
The availability of millions of games at one's
fingertips in a database is also making the game's best players younger and
younger. Absorbing the thousands of essential patterns and opening moves
used to take many years, a process indicative of Malcolm Gladwell's "10,000
hours to become an expert" theory as expounded in his recent book
Outliers. (Gladwell's earlier book, Blink, rehashed, if more
creatively, much of the cognitive psychology material that is re-rehashed in
Chess Metaphors.) Today's teens, and increasingly pre-teens, can
accelerate this process by plugging into a digitized archive of chess
information and making full use of the superiority of the young mind to
retain it all. In the pre-computer era, teenage grandmasters were rarities
and almost always destined to play for the world championship. Bobby
Fischer's 1958 record of attaining the grandmaster title at fifteen was
broken only in 1991. It has been broken twenty times since then, with the
current record holder, Ukrainian Sergey Karjakin, having claimed the highest
title at the nearly absurd age of twelve in 2002. Now twenty, Karjakin is
among the world's best, but like most of his modern wunderkind peers he's no
Fischer, who stood out head and shoulders above his peers—and soon enough
above the rest of the chess world as well.
Excelling at chess has long been
considered a symbol of more general intelligence. That is an incorrect
assumption in my view, as pleasant as it might be. But for the purposes of
argument and investigation, chess is, in Russkin-Gutman's words, "an
unparalleled laboratory, since both the learning process and the degree of
ability obtained can be objectified and quantified, providing an excellent
comparative framework on which to use rigorous analytical techniques."
Here I agree wholeheartedly, if for different
reasons. I am much more interested in using the chess laboratory to
illuminate the workings of the human mind, not the artificial mind. As I put
it in my 2007 book, How Life Imitates Chess, "Chess is a unique
cognitive nexus, a place where art and science come together in the human
mind and are then refined and improved by experience." Coincidentally the
section in which that phrase appears is titled "More than a metaphor." It
makes the case for using the decision-making process of chess as a model for
understanding and improving our decision-making everywhere else.
This is not to say that I am not interested in the
quest for intelligent machines. My many exhibitions with chess computers
stemmed from a desire to participate in this grand experiment. It was my
luck (perhaps my bad luck) to be the world chess champion during the
critical years in which computers challenged, then surpassed, human chess
players. Before 1994 and after 2004 these duels held little interest. The
computers quickly went from too weak to too strong. But for a span of ten
years these contests were fascinating clashes between the computational
power of the machines (and, lest we forget, the human wisdom of their
programmers) and the intuition and knowledge of the grandmaster.
In what Rasskin-Gutman explains as Moravec's
Paradox, in chess, as in so many things, what computers are good at is where
humans are weak, and vice versa. This gave me an idea for an experiment.
What if instead of human versus machine we played as partners? My brainchild
saw the light of day in a match in 1998 in León, Spain, and we called it
"Advanced Chess." Each player had a PC at hand running the chess software of
his choice during the game. The idea was to create the highest level of
chess ever played, a synthesis of the best of man and machine.
Although I had prepared for the unusual format, my
match against the Bulgarian Veselin Topalov, until recently the world's
number one ranked player, was full of strange sensations. Having a computer
program available during play was as disturbing as it was exciting. And
being able to access a database of a few million games meant that we didn't
have to strain our memories nearly as much in the opening, whose
possibilities have been thoroughly catalogued over the years. But since we
both had equal access to the same database, the advantage still came down to
creating a new idea at some point.
Having a computer partner also meant never having
to worry about making a tactical blunder. The computer could project the
consequences of each move we considered, pointing out possible outcomes and
countermoves we might otherwise have missed. With that taken care of for us,
we could concentrate on strategic planning instead of spending so much time
on calculations. Human creativity was even more paramount under these
conditions. Despite access to the "best of both worlds," my games with
Topalov were far from perfect. We were playing on the clock and had little
time to consult with our silicon assistants. Still, the results were
notable. A month earlier I had defeated the Bulgarian in a match of
"regular" rapid chess 4–0. Our advanced chess match ended in a 3–3 draw. My
advantage in calculating tactics had been nullified by the machine.
This experiment goes unmentioned by Russkin-Gutman,
a major omission since it relates so closely to his subject. Even more
notable was how the advanced chess experiment continued. In 2005, the online
chess-playing site Playchess.com hosted what it called a "freestyle" chess
tournament in which anyone could compete in teams with other players or
computers. Normally, "anti-cheating" algorithms are employed by online sites
to prevent, or at least discourage, players from cheating with computer
assistance. (I wonder if these detection algorithms, which employ diagnostic
analysis of moves and calculate probabilities, are any less "intelligent"
than the playing programs they detect.)
Lured by the substantial prize money, several
groups of strong grandmasters working with several computers at the same
time entered the competition. At first, the results seemed predictable. The
teams of human plus machine dominated even the strongest computers. The
chess machine Hydra, which is a chess-specific supercomputer like Deep Blue,
was no match for a strong human player using a relatively weak laptop. Human
strategic guidance combined with the tactical acuity of a computer was
overwhelming.
The surprise came at the conclusion of the event.
The winner was revealed to be not a grandmaster with a state-of-the-art PC
but a pair of amateur American chess players using three computers at the
same time. Their skill at manipulating and "coaching" their computers to
look very deeply into positions effectively counteracted the superior chess
understanding of their grandmaster opponents and the greater computational
power of other participants. Weak human + machine + better process was
superior to a strong computer alone and, more remarkably, superior to a
strong human + machine + inferior process.
Continued in article
"Critical Thinking: Why It's So Hard to Teach," by Daniel T.
Willingham ---
http://www.aft.org/pubs-reports/american_educator/issues/summer07/Crit_Thinking.pdf
Also see Simorleon Sense ---
http://www.simoleonsense.com/critical-thinking-why-is-it-so-hard-to-teach/
“Critical thinking is not a set of
skills that can be deployed at any time, in any context. It is a type of
thought that even 3-year-olds can engage in—and even trained scientists
can fail in.”
“Knowing that one should think
critically is not the same as being able to do so. That requires domain
knowledge and practice.”
So, Why Is Thinking Critically So
Hard?
Educators have long noted that school attendance and even academic
success are no guarantee that a student will graduate an effective
thinker in all situations. There is an odd tendency for rigorous
thinking to cling to particular examples or types of problems. Thus, a
student may have learned to estimate the answer to a math problem before
beginning calculations as a way of checking the accuracy of his answer,
but in the chemistry lab, the same student calculates the components of
a compound without noticing that his estimates sum to more than 100
percent. And a student who has learned to thoughtfully discuss the
causes of the American Revolution from both the British and American
perspectives doesn’t even think to question how the Germans viewed World
War II. Why are students able to think critically in one situation, but
not in another? The brief answer is: Thought processes are intertwined
with what is being thought about. Let’s explore this in depth by looking
at a particular kind of critical thinking that has been studied
extensively: problem solving.
Imagine a seventh-grade math class immersed in
word problems. How is it that students will be able to answer one
problem, but not the next, even though mathematically both word problems
are the same, that is, they rely on the same mathematical knowledge?
Typically, the students are focusing on the scenario that the word
problem describes (its surface structure) instead of on the mathematics
required to solve it (its deep structure). So even though students have
been taught how to solve a particular type of word problem, when the
teacher or textbook changes the scenario, students still struggle to
apply the solution because they don’t recognize that the problems are
mathematically the same.
Thinking Tends to Focus on a Problem’s
“Surface Structure”
To understand why the surface structure of a problem is so distracting
and, as a result, why it’s so hard to apply familiar solutions to
problems that appear new, let’s first consider how you understand what’s
being asked when you are given a problem. Anything you hear or read is
automatically interpreted in light of what you already know about
similar subjects. For example, suppose you read these two sentences:
“After years of pressure from the film and television industry, the
President has filed a formal complaint with China over what U.S. firms
say is copyright infringement. These firms assert that the Chinese
government sets stringent trade restrictions for U.S. entertainment
products, even as it turns a blind eye to Chinese companies that copy
American movies and television shows and sell them on the black market.”
With Deep Knowledge, Thinking Can
Penetrate Beyond Surface Structure
If knowledge of how to solve a problem never transferred to problems
with new surface structures, schooling would be inefficient or even
futile—but of course, such transfer does occur. When and why is
complex,5 but two factors are especially relevant for educators:
familiarity with a problem’s deep structure and the knowledge that one
should look for a deep structure. I’ll address each in turn. When one is
very familiar with a problem’s deep-structure, knowledge about how to
solve it transfers well. That familiarity can come from long-term,
repeated experience with one problem, or with various manifestations of
one type of problem (i.e., many problems that have different surface
structures, but the same deep structure). After repeated exposure to
either or both, the subject simply perceives the deep structure as part
of the problem description.
"Beyond Critical Thinking," by Michael S. Roth, Chronicle of Higher
Education's Chronicle Review, January 3, 2010 ---
http://chronicle.com/article/Beyond-Critical-Thinking/63288/
The antivocational dimension of the
humanities has been a source of pride and embarrassment for generations. The
persistence of this reputed uselessness is puzzling given the fact that an
education in the humanities allows one to develop skills in reading,
writing, reflection, and interpretation that are highly prized in our
economy and culture. Sure, specific training in a discrete set of skills
might prepare you for Day 1 of the worst job you'll ever have (your first),
but the humanities teach elements of mind and heart that you will draw upon
for decades of innovative and focused work. But we do teach a set of skills,
or an attitude, in the humanities that may have more to do with our
antipractical reputation than the antivocational notion of freedom embedded
in the liberal arts. This is the set of skills that usually goes under the
rubric of critical thinking.
Although critical thinking first gained
its current significance as a mode of interpretation and evaluation to guide
beliefs and actions in the 1940s, the term took off in education circles
after Robert H. Ennis published "A Concept of Critical Thinking" in the
Harvard Educational Review in 1962. Ennis was interested in how we teach the
"correct assessment of statements," and he offered an analysis of 12 aspects
of this process. Ennis and countless educational theorists who have come
after him have sung the praises of critical thinking. There is now a
Foundation for Critical Thinking and an industry of consultants to help you
enhance this capacity in your teachers, students, or yourself.
A common way to show that one has
sharpened one's critical thinking is to display an ability to see through or
undermine statements made by (or beliefs held by) others. Thus, our best
students are really good at one aspect of critical thinking—being critical.
For many students today, being smart means being critical. To be able to
show that Hegel's concept of narrative foreclosed the non-European, or that
Butler's stance on vulnerability contradicts her conception of
performativity, or that a tenured professor has failed to account for his
own "privilege"—these are marks of sophistication, signs of one's ability to
participate fully in the academic tribe. But this participation, being
entirely negative, is not only seriously unsatisfying; it is ultimately
counterproductive.
The skill at unmasking error, or simple
intellectual one-upmanship, is not completely without value, but we should
be wary of creating a class of self-satisfied debunkers or, to use a
currently fashionable word on campuses, people who like to "trouble" ideas.
In overdeveloping the capacity to show how texts, institutions, or people
fail to accomplish what they set out to do, we may be depriving students of
the capacity to learn as much as possible from what they study. In a
humanities culture in which being smart often means being a critical
unmasker, our students may become too good at showing how things don't make
sense. That very skill may diminish their capacity to find or create meaning
and direction in the books they read and the world in which they live. Once
outside the university, our students continue to score points by displaying
the critical prowess for which they were rewarded in school. They wind up
contributing to a cultural climate that has little tolerance for finding or
making meaning, whose intellectuals and cultural commentators delight in
being able to show that somebody else is not to be believed.
I doubt that this is a particularly
contemporary development. In the 18th century there were complaints about an
Enlightenment culture that prized only skepticism and that was satisfied
only with disbelief. Our contemporary version of this trend, though, has
become skeptical even about skepticism. We no longer have the courage of our
lack of conviction. Perhaps that's why we teach our students that it's cool
to say that they are engaged in "troubling" an assumption or a belief. To
declare that one wanted to disprove a view would show too much faith in the
ability to tell truth from falsehood. And to declare that one was receptive
to learning from someone else's view would show too much openness to being
persuaded by an idea that might soon be deconstructed (or simply mocked).
In training our students in the techniques
of critical thinking, we may be giving them reasons to remain guarded—which
can translate into reasons not to learn. The confident refusal to be
affected by those with whom we disagree seems to have infected much of our
cultural life: from politics to the press, from siloed academic programs (no
matter how multidisciplinary) to warring public intellectuals. As humanities
teachers, however, we must find ways for our students to open themselves to
the emotional and cognitive power of history and literature that might
initially rub them the wrong way, or just seem foreign. Critical thinking is
sterile without the capacity for empathy and comprehension that stretches
the self.
One of the crucial tasks of the humanities
should be to help students cultivate the willingness and ability to learn
from material they might otherwise reject or ignore. This material will
often surprise students and sometimes upset them. Students seem to have
learned that teaching-evaluation committees take seriously the criticism
that "the professor, or the material, made me uncomfortable." This complaint
is so toxic because being made uncomfortable may be a necessary component of
an education in the humanities. Creating a humanistic culture that values
the desire to learn from unexpected and uncomfortable sources as much as it
values the critical faculties would be an important contribution to our
academic and civic life.
But the contemporary humanities should do
more than supplement critical thinking with empathy and a desire to
understand others from their own point of view. We should also supplement
our strong critical engagement with cultural and social norms by developing
modes of teaching that allow our students to enter in the value-laden
practices of a particular culture to understand better how these values are
legitimated: how the values are lived as legitimate. Current thinking in the
humanities is often strong at showing that values that are said to be shared
are really imposed on more-vulnerable members of a particular group. Current
thinking in the humanities is also good at showing the contextualization of
norms, whether the context is generated by an anthropological, historical,
or other disciplinary matrix. But in both of these cases we ask our students
to develop a critical distance from the context or culture they are
studying.
Many humanities professors have become
disinclined to investigate with our students how we generate the values we
believe in, or the norms according to which we go about our lives. In other
words, we have been less interested in showing how we make a norm legitimate
than in sharpening our tools for delegitimization. The philosopher Robert
Pippin has recently made a similar point, and has described how evolutionary
biology and psychology have moved into this terrain, explaining moral values
as the product of the same dynamic that gives rise to the taste for sweets.
Pippin argues, on the contrary, that "the practical autonomy of the
normative is the proper terrain of the humanities," and he has an easy task
of showing how the pseudoscientific evolutionary "explanation" of our moral
choices is a pretty flimsy "just-so" story.
If we humanities professors saw ourselves
more often as explorers of the normative than as critics of normativity, we
would have a better chance to reconnect our intellectual work to broader
currents in public culture. This does not have to mean an acceptance of the
status quo, but it does mean an effort to understand the practices of
cultures (including our own) from the point of view of those participating
in them. This would include an understanding of how cultures change. For
many of us, this would mean complementing our literary or textual work with
participation in community, with what are often called service-learning
courses. For others, it would mean approaching our object of study not with
the anticipated goal of exposing weakness or mystification but with the goal
of turning ourselves in such a way as to see how what we study might inform
our thinking and our lives.
I realize that I am arguing for a mode of
humanistic education that many practice already. It is a mode that can take
language very seriously, but rather than seeing it as the master mediator
between us and the world, a matrix of representations always doomed to fail,
it sees language as itself a cultural practice to be understood from the
point of view of those using it.
The fact that language fails according to
some impossible criterion, or that we fail in our use of it, is no news,
really. It is part of our finitude, but it should not be taken as the key
marker of our humanity. The news that is brought by the humanities is a way
of turning the heart and the spirit so as to hear possibilities of various
forms of life in which we might participate. When we learn to read or look
or listen intensively, we are not just becoming adept at exposing falsehood
or at uncovering yet more examples of the duplicities of culture and
society. We are partially overcoming our own blindness by trying to
understand something from another's artistic, philosophical, or historical
point of view. William James put it perfectly in a talk to teachers and
students entitled "On a Certain Blindness in Human Beings": "The meanings
are there for others, but they are not there for us." James saw the
recognition of this blindness as key to education as well as to the
development of democracy and civil society. Of course hard-nosed critical
thinking may help in this endeavor, but it also may be a way we learn to
protect ourselves from the acknowledgment and insight that humanistic study
has to offer. As students and as teachers we sometimes crave that protection
because without it we risk being open to changing who we are. In order to
overcome this blindness, we risk being very uncomfortable indeed.
It is my hope that humanists will continue
offering criticism, making connections, and finding ways to acknowledge
practices that seem at first opaque or even invisible. In supporting a
transition from critical thinking to practical exploration, I am echoing a
comment made by my undergraduate philosophy teacher Louis Mink, and echoed
by my graduate mentor, Richard Rorty. Years before Dick Rorty deconstructed
the idea of the "philosopher as referee," Louis Mink suggested that critics
"exchange the judge's wig for the guide's cap." I think we may say the same
for humanists, who can, in his words, "show us details and patterns and
relations which we would not have seen or heard for ourselves."
My humanities teachers enriched my life by
showing me details and pattern and relations. In so doing they also helped
me to acquire tools that have energetically shaped my scholarship and my
interactions with colleagues and students. It is my hope that as guides, not
judges, we can show our students how to engage in the practice of exploring
objects, norms, and values that inform diverse cultures. In doing so,
students will develop the ability to converse with others about shaping the
objects, norms, and values that will inform their own lives. They will
develop the ability to add value to (and not merely criticize values in)
whatever organizations in which they participate. They will often reject
roads that others have taken, and they will sometimes chart new paths. But
guided by the humanities, they will increase their ability to find together
ways of living that have meaning and direction, illuminating paths immensely
practical and sustaining.
Michael S. Roth is an intellectual historian and
president of Wesleyan University. This essay was part of a lecture
commemorating the 50th anniversary of the founding of Wesleyan's Center for
the Humanities.
January 5, 2009 reply from Barbara Scofield
[barbarawscofield@GMAIL.COM]
At the University of Kentucky in the 1990s I took a
faculty development course in "Integrative Studies," which was required of
the medical students at that time, and then offered one summer to all
faculty. In the discussion segments the faculty participants were asked to
always provide comments that were an addition to the comments of the other
participants. In other words, we couldn't begin with "Yes, but ..." We were
supposed to find common ground and build from there. Some faculty found this
impossible to do, even when the facilitator emphasized it over and over
again. My remembrance is that the business and agriculture faculty had an
easier time with the cooperative nature of the course than the liberal arts
folks.
Barbara W. Scofield, PhD, CPA Chair of Graduate
Business Studies Professor of Accounting The University of Texas of the
Permian Basin 4901 E. University Dr. Odessa, TX 79762
432-552-2183 (Office) 817-988-5998 (Cell)
BarbaraWScofield@gmail.com
The Cambridge Handbook of Thinking and Reasoning
---
Click Here
The Miniature Guide To Critical Thinking Concepts & Tools ---
Click Here
Bob Jensen's threads on critical thinking are at
http://faculty.trinity.edu/rjensen/HigherEdControversies.htm#CriticalThinking
College Degrees Without Instructors
Competency-Based Assessment ---
http://faculty.trinity.edu/rjensen/competency.htm
There are a few really noteworthy competency-based distance education
programs including Western Governors University (WGU) and the Chartered
Accountancy School of Business (CASB) in Canada. But these
competency-based programs typically have assigned instructors and bear the costs
of those instructors. The instructors, however, do not assign grades to
students.
It appears that the Southern New Hampshire University (a private institution)
is taking competency-based distance education to a new level by eliminating the
instructors. It should be noted that SNHU has both an onsite campus and online
degree programs.
"Online Education Is Everywhere. What’s the Next Big Thing?" by Marc
Parry, Chronicle of Higher Education, August 31, 2011 ---
http://chronicle.com/blogs/wiredcampus/online-education-is-everywhere-whats-the-next-big-thing/32898?sid=wc&utm_source=wc&utm_medium=en
. . .
The vision is that students could sign up for
self-paced online programs with no conventional instructors. They could work
at their own speeds through engaging online content that offers built-in
assessments, allowing them to determine when they are ready to move on. They
could get help through networks of peers who are working on the same
courses; online discussions could be monitored by subject experts. When
they’re ready, students could complete a proctored assessment, perhaps at a
local high school, or perhaps online. The university’s staff could then
grade the assessment and assign credit.
And the education could be far cheaper, because
there would be no expensive instructor and students could rely on free, open
educational resources rather than expensive textbooks. Costs to the student
might include the assessment and the credits.
“The whole model hinges on excellent assessment, a
rock-solid confidence that the student has mastered the student-learning
outcomes,” the memo says. “If we know with certainty that they have, we
should no longer care if they raced through the course or took 18 months, or
if they worked on their courses with the support of a local church
organization or community center or on their own. The game-changing idea
here is that when we have assessment right, we should not care how a student
achieves learning. We can blow up the delivery models and be free to try
anything that shows itself to work.”
Continued in article
"A Russian University Gets Creative Against Corruption: With
surveillance equipment and video campaigns, rector aims to eliminate bribery at
Kazan State," by Anna Nemtsova, Chronicle of Higher Education, January
17, 2010 ---
http://chronicle.com/article/A-Russian-University-Gets/63522/
Jensen Comment
In its early history, the University of Chicago had competency-based programs
where grades were assigned solely on the basis of scores on final examinations.
Students did not have to attend class.
Bob Jensen's threads on competency-based assessment ---
http://faculty.trinity.edu/rjensen/competency.htm
Bob Jensen's threads on distance education alternatives are at
http://faculty.trinity.edu/rjensen/Crossborder.htm
Bob Jensen's threads on higher education controversies are at
http://faculty.trinity.edu/rjensen/HigherEdControversies.htm
I should point out that this is very similar to the AAA's Innovation in
Accounting Education Award Winning BAM Pedagogy commenced at the University of
Virginia (but there were instructors who did not teach) ---
http://faculty.trinity.edu/rjensen/265wp.htm
Appendix 13
Updates
Metacognition in Learning
To my knowledge, Bob Jensen is the first author to discuss the importance of
metacognition in learning.
That paper focuses on the metacognitive advantages of
self-learning (with blood, sweat, and tears) over memorizing answers given out
by teachers.
"Metacognitive Concerns in Designs and Evaluations of Computer Aided
Education and Training: Are We Misleading Ourselves About Measures of Success?"
http://faculty.trinity.edu/rjensen/265wp.htm
Now we have a second paper on he importance of metacognition in learning
The paper below focuses on the metacognitive mindset
"Accounting Students' Metacognition: The Association of Performance,
Calibration Error, and Mindset," by Susan P. Ravenscroft, Tammy R. Waymire,
and Timothy D. West, Issues in Accounting Education, Vol. 27, No. 3, August
2012, pp. 707-732 (not free) ---
http://aaajournals.org/doi/full/10.2308/iace-50148
In recognition of the evolving body of knowledge in
the accounting profession, the American Institute of Certified Public
Accountants (AICPA 2010) highlights the importance of viewing learning as a
lifelong process that requires self-awareness and extends beyond the
academic setting. Metacognition, the assessment and regulation of one's own
learning, is a crucial element in lifelong learning. We draw upon judgment
of learning research and introduce mindset theory to explore the
relationship among (1) exam performance, (2) calibration error, measured as
expected minus actual exam scores, and (3) mindset, a person's basic beliefs
about learning and ability (Dweck 2000, 2006) in the accounting classroom.
We find strong evidence that exam performance is inversely related to
calibration error (Kruger and Dunning 1999). We also find modest evidence
that a growth mindset is associated with improved performance and decreased
calibration error. While the mindset results were not entirely consistent
with prior research in educational psychology, we explore possible reasons
and future directions for accounting education research.
. . .
DISCUSSION Limitations
Our sample consisted of students taught by a single
instructor at a single institution who took an elective governmental and
nonprofit accounting course during one of three semesters. This course is
typically viewed as difficult and as needed for the CPA examination. While
this could restrict the generalizability of the results, we do not believe
that it does so seriously. We are aware of no research findings indicating
that judgments of learning or mindsets differ across social demographics.
Instead, the findings on which we relied are found across broad categories
of groups. However, to establish generalizability, we hope to use multiple
institutions, instructors, and courses in future research.
Another limitation is the restriction of range that
we found in the independent variable of mindset. Dweck and Molden (2005)
note that when they assess children or adults, they find that about 40
percent endorse the fixed view of mindset, another 40 percent endorse the
growth view, and about 20 percent are undecided. Given that a majority of
the subjects were categorized as having a growth mindset, the likelihood of
seeing a significant relationship was decreased. Because we did not
manipulate this variable, we could not create a full range of mindsets for
our analysis. Moreover, we have a restricted range of performance. Students
taking the governmental and nonprofit accounting course have all succeeded
in a competitive accounting program, with average GPAs above that required
for remaining the program. Both of these restrictions bias against finding
statistically significant relationships, and we believe that the results
can, therefore, still be of benefit to a broad range of accounting
educators. Discussion of Results
The initial goal of this study was to better
understand why accounting students sometimes lack self-awareness about their
own abilities and skills, and to explore factors that may assist accounting
educators. The study's results point to three implications for accounting
educators. First, consistent with Kruger and Dunning (1999), we found that
students who overestimate their abilities likely do so because they lack the
technical knowledge to evaluate their own performance, as evidenced by lower
performance. We also found, in the first two exams, evidence of a magnitude
effect that suggests that high-performing students calibrate more accurately
than low-performing students do, expressed in absolute terms. This may
affirm observations by accounting faculty and help them in assisting
students with their self-regulated learning and self-insight.
Second, in exploring the average calibration errors
of high- and low-performing students, we found that low-performing students
tend to improve their calibration accuracy, while high-performing students
tend to become increasingly underconfident relative to their performance.
These results demonstrate the concerns that accounting educators may have
for both low performers and high performers. Low performers' lack the
self-awareness of their technical skills to accurately calibrate their own
performance, and this may cause them to continue to underperform. High
performers fail to recognize their strong technical skills and may become
overly critical of their own performance.
Third, in exploring the role of mindset regarding
an individual's approach to learning and response to failure, we predicted
that students with a growth mindset (i.e., those who were motivated by
learning, resilient, and focused on learning from feedback) would
demonstrate higher exam performance, improvement in performance, lower
calibration error, and improvement in their calibration. We find modest
evidence supporting these predictions. Mindset was significantly associated
with performance on only one of three exams, and improvement from Exam 1 to
Exam 2. Mindset was not associated with level of calibration error, but was
associated with improvement in calibration from Exam 1 to Exam 2 and from
Exam 1 to Exam 3. We expected growth mindset to be more consistently
associated with the level of, and improvements in, calibration error;
however, we believe that the short, one-semester timeline may make it more
difficult to capture the impact of mindset. Furthermore, we present evidence
that the final exam (Exam 3) may reflect unique resource allocation
decisions on the part of students that may affect both the performance and
calibration error results. Although inconsistent, the results provide some
modest evidence that encouraging a growth mindset may offer benefits to
students in improving their performance and calibration accuracy.
Mindset theory originated as a way to explain why
students have differing goals and reactions to failure (Dweck and Leggett
1988), but as the research in this area has continued, the significance and
implications of mindset have grown. For instance, more recent work implies
that mindsets—although malleable experimentally—represent a fundamental view
of the world, quoting Piaget to the effect that worldviews of children “can
be as important to their functioning as the logical reasoning he studied for
much of his career,” (Molden and Dweck 2006, 200). Molden and Dweck (2006)
survey research showing that mindset plays a role in many behaviors,
including goal setting, attributions, strategies, grades, perceptions of
others, responses to stereotypes, self-esteem, and self-regulatory
strategies.
In our setting, senior-level and graduate
accounting students who have met stringent admissions criteria and who are
very grade-conscious may hold strong achievement goals. The connection
between mindset and performance may be altered in the presence of strongly
held achievement goals (grade-based as opposed to learning-based). Dweck and
other researchers (e.g., Dweck and Leggett 1988; Shunk 1995) observe that
the positive effect of mindset on achievement can be overridden by the
effect of goals. Shunk (1995, 317) discusses the interaction of goals and
mindset, and notes that sometimes “success-oriented persons who perform
poorly on one occasion will work harder and improve their performance on
another.” The integration of the goals literature may, therefore, be helpful
in future exploration of the role of mindset in the accounting education
setting and extending the results presented in this study. Furthermore,
because research suggests that business students generally approach studying
in a more superficial way than non-business students (Arum and Roska 2011),
future research studies could be conducted across academic disciplines,
preferably including students in and outside the college of business to make
comparisons among groups.
In sum, our study presents evidence of an inverse
relationship between performance and calibration error in an accounting
education setting, and offers an initial step in understanding the role
mindset plays in metacognitive self-awareness of accounting students.
Although this research represents an early effort to introduce mindset
concepts within the accounting education literature, our results and the
underlying research suggest that faculty could introduce the concept of
mindset to students, which could be particularly useful for those students
with fixed mindsets. Introducing the concept of a growth mindset leads
naturally into a discussion of the effort that is necessary for deep
learning, and could motivate a discussion with student involvement about the
students' study approaches and preparation for tests. Finally, recent
research (Anseel et al. 2009) suggests that the beneficial effects of
faculty feedback to students can be amplified if students are appropriately
guided to reflect on their performance. Mindset, in conjunction with
feedback, offers promise as a way to encourage learning and self-awareness.
"Designing for Emergence: The Role of the Instructor in Student-Centered
Learning," by Mary Stewart, Hybrid Pedagogy, August 21, 2014 ---
http://www.hybridpedagogy.com/journal/designing-emergence-role-instructor-student-centered-learning/
Jensen Comment
I seldom say that any of my tidbits are "must reads." This is an exception. I
think it is a must read for all students and faculty!
"Book Review: 'The Organized Mind' by Daniel J. Levitin --- Our minds were
designed to succeed in an environment utterly unlike the information overload we
now face," by Christopher Chabris, The Wall Street Journal, August
15, 2014 ---
http://online.wsj.com/articles/book-review-the-organized-mind-by-daniel-j-levitin-1408137852?tesla=y&mod=djemMER_h&mg=reno64-wsj
Dr. Chabris is a psychology professor at Union College and co-author,
with Daniel Simons, of "The Invisible Gorilla: How Our Intuitions Deceive
Us."
More than a century ago, Sigmund Freud wrote the
"Psychopathology of Everyday Life." Over two decades ago, Donald Norman
published the "Psychology of Everyday Things." Three years ago, David Myers
called a new edition of his textbook "Psychology in Everyday Life." The word
"everyday" has a special appeal in such titles, since so many psychology
books, especially of the self-help variety, are written for the self with
major problems to contend with—love, illness, grief, identity,
conflict—leaving the small tasks of mundane functioning to common sense, or
perhaps to business writers who purvey "habits" and "disciplines."
In "The Organized Mind," Daniel J. Levitin, a
cognitive neuroscientist at McGill University, makes an ambitious attempt to
bring research in neuroscience and cognitive psychology to bear on the more
ordinary parts of our lives. He focuses on the daily challenges of
professionals, managers and knowledge workers. But we are all knowledge
workers now, since everyone uses Facebook, FB -0.90% communicates by email,
and must process, store and retrieve an ever-growing volume of information.
In this impressively wide-ranging and thoughtful work, Mr. Levitin stresses
the many ways in which evolution designed our minds to succeed in an
environment that was utterly unlike the world of information overload we now
face. And he aims to help us cope by providing concrete suggestions for
solving the daily problems of modern existence.
Mr. Levitin begins by explaining why we are in the
mess we are in. The capacities of our brains grew out of solutions to the
problems that our ancestor species confronted when living in the natural
world. We have very good memories for routes we walk and for places where
things are located because those are the most important things for primates
and mammals to be keep track of. And our tendency to be attracted by
anything new had great value when new things were likely to be important
threats or opportunities. But these capacities may be maladapted to the
challenges of current life, especially the man-made parts of it.
Memories tuned for routes and places are simply not
designed to store the near-infinity of unique passwords (random strings of
letters, numbers and punctuation) that Internet security demands.
Decision-making systems that put a premium on novelty betray us when
millions upon millions of new data packets are mere finger-taps away. "Every
status update you read on Facebook, every tweet or text message you get from
a friend," Mr. Levitin writes, "is competing for resources in your brain
with important things like whether to put your savings in stocks or bonds,
where you left your passport, or how to reconcile with a close friend you
just had an argument with." All this piling up taxes our abilities to
process information, remember it and make decisions.
What to do? Mr. Levitin devotes several meaty
chapters to specific domains—including domestic matters, social connections
and time management—in which we tend to fall short of what is needed for
peace of mind and productivity. He also considers how to teach younger
people to cope with the information-rich environment they will grow up in.
Throughout, he mixes anecdote and science, first-person narrative and tips
for successful living. On one page you may read a detailed description of a
brain-imaging study, on the next a quotation from the man who was once in
charge of managing President Obama's mail, and after that a formula for
generating strong but memorable passwords.
A good way to deal with overload, Mr. Levitin
suggests, is to offload the responsibilities of "personal management," tasks
like being on time and staying in touch with friends or associates. This
strategy is routinely adopted by members of a category that Mr. Levitin
calls HSPs—Highly Successful Persons. A few months ago, I was excited to
learn that my newest Twitter TWTR -1.26% followers were Phil Ivey, one of
the best poker players in the world, and the popular actor Taye Diggs —both
undoubted HSPs. Then I found out that they employ other people to manage
their social media. Mr. Levitin tells us that he met Jimmy Carter back in
the mid-1970s, when he was first running for president, and Mr. Carter spoke
"as though we had all the time in the world." He could focus on the task at
hand, Mr. Levitin notes, because his aides were worrying about where he
needed to be and when, freeing him to "let go of those inner nagging voices
and be there."
Mr. Levitin isn't recommending that we all hire
personal assistants, an unrealistic approach unless you happen to be an HSP
or a Real Housewife. We don't need human helpers because computational ones
become better all the time. A calendar app that buzzes quietly 15 minutes
before each appointment is better and cheaper than a human who has to knock
on your door and interrupt your conversation or train of thought. Sites like
Orbitz and Kayak are faster and more flexible for booking almost any trip
than human travel agents ever were. A well-curated Twitter feed will keep
you up on news about your work and hobbies in a way that no personal
assistant ever could. Indeed, we have outsourced to Google GOOGL -0.16% a
massive volume of "research" chores that used to take anywhere from minutes
to months of trawling through reference sources, making phone calls and
visiting archives. These conveniences come with frustrations (also known as
"first-world problems"), but to focus on such trees is to miss the forest of
improvements that we enjoy today.
To a surprising extent, Mr. Levitin's advice for
organizing our minds consists not of learning mental tricks or doing brain
exercises but of organizing our surroundings—literally, the physical world
we inhabit every day. This effort can reduce needless demands on our
cognitive abilities, especially on our capacity for paying attention, which
he rightly calls "the most essential mental resource for any organism." The
suggestions range from the simple (keep in plain sight the things you need
to access most often) to the detailed, such as setting up various filing
systems, including a junk drawer and "miscellaneous" file.
Explaining filing systems, Mr. Levitin writes, "the
key to creating useful categories in our homes is to limit the number of
types of things they contain to one or at most four (respecting the capacity
limitations of working memory)." By keeping items that share a common use in
a single place (e.g., supplies for a party) you reduce the burden on memory,
since you must only associate one location ("top middle kitchen island
drawer") with one category ("birthday party") rather than several locations
(different drawers and cabinets) with several items (colored napkins, paper
plates, plastic cups). The same idea applies to files of important documents
and media, whether physical or digital.
In the case of the junk drawer, Mr. Levitin
explains that, while we have a powerful instinct to categorize things, we
don't benefit from creating categories that have only a single member.
Uncategorizable stuff should go together in its own space because this is
the easiest way for the human mind to keep track of it. If you have a lot of
folders with just one document in each, you might soon forget that those
folders even exist. It would be better if their contents remained together
in a bigger "misc" folder.
When it comes to teaching younger generations how
to organize their minds, Mr. Levitin makes some surprising points. Today's
college students are thought of as "digital natives" who are inherently
skilled with computers and the Internet because they grew up with them from
their earliest days. But they have trouble distinguishing media outlets and
websites that at least try to report news and facts objectively from those
that are deliberately partisan or ideological. Even medical students aren't
good at telling high-quality journals (whose research reports should be
given more trust) from low-quality ones (whose reports should be regarded
with skepticism). Given that our minds mostly evolved long before the
invention of reading and writing, let alone mass media, it stands to reason
that a fine eye for evaluating the quality of sources must be learned, and
even taught, rather than assumed to be part of our standard equipment.
What to do about this? It turns out that librarians
have already responded by writing guides to evaluating sources. These
include questions like "is the page current?" and "what is the domain?" (A
page from nih.gov will have more authoritative medical advice than one from
autismspeaks.org.) These suggestions point to considerations that many of us
take for granted but that are increasingly crucial for everyone to grasp,
now that Google's cornucopia is rarely farther away than our hip pockets.
Some of Mr. Levitin's recommendations may seem like
little more than common sense or reiterations of techniques developed from
generations of experience. Has anyone not heard of a junk drawer? But if
there is one lesson to be drawn from the past century of research on human
behavior it is that common sense is a much poorer guide to life than, well,
common sense would have us believe. Common sense is often contradicted by
empirical evidence—we assume, for instance, that we remember important
events in precise detail, when research shows that such memories become
distorted and decay over time. And elements of common sense frequently
contradict one another. How can opposites attract but birds of a feather
flock together? When common sense can be shown to be consistent with solid
scientific principles, we should prize it all the more.
"The Organized Mind" is an organized book, but it
also rewards dipping in at any point, for there are fascinating facts and
examples throughout. Mr. Levitin concisely explains Kolmogorov complexity (a
way to measure how much information a message or algorithm contains, which
can help with communicating and storing data optimally). He provides a handy
inventory of things to keep extras of in your luggage, so you don't have to
remember them and then scurry around to gather them up before you leave for
a trip. (Most important, keep a phone charger in your bag at all times.) He
even lists the rules that govern how interstate highways are numbered—with
examples and a map. The point is that if you learn the system, you don't
need to memorize the specific directions and junctions of dozens of
individual highways. (Nowadays, though, we outsource this knowledge to
GPS-equipped cars and phones.) An appendix explains how to construct simple
2x2 tables to properly interpret important percentages and probabilities
like the likelihood of having a serious disease given a positive diagnostic
test.
Like any neuroscience-based book, "The Organized
Mind" has to confront the problem of the still-tentative nature of many of
the most fascinating findings and resist the ever-present temptation to pick
a few new ideas to weave a just-so story. Books with titles like "the new
science of X" or "the neuroscience of Y" can almost be counted on to be
wrong—often breathlessly so. Mr. Levitin mostly eludes this trap by sticking
to established principles, such as the limited capacities of memory and
attention or the biases that plague our thinking (e.g., the difficulty of
reasoning rationally about risks). None of these principles will rise or
fall at the next conference or in the next edition of a scholarly journal.
Clearly, most of us could use more organization in
our lives and in our minds. But does a push for organization and focus, for
the optimal use of our scarce attention, come at any cost? Perhaps. Students
who use "study drugs" like Ritalin or Adderall to stay awake and concentrate
on work and deadlines report that they find it easier to complete projects
but feel that the results are less creative than they otherwise would be.
Such a tradeoff—if it is even more than anecdotal—could be the natural
result of "powering through" a task in a fixed time frame, regardless of
whether drugs or other forces were involved.
But such trade-offs do not mean that organization
and creativity are enemies. Without enough organization to complete a
project, no amount of creativity will have an effect. And some of the most
creative people, Mr. Levitin notes, were also some of the most fanatical
organizers of their output. Michael Jackson employed a full-time archivist,
and John Lennon "kept boxes and boxes of work tapes of songs in progress,
carefully labeled." Perhaps this habit explains why the ex-Beatle seemed to
go on releasing album after album of new material after his death. In any
case, it's a mistake to think creativity suffers from organization, that a
messy desk or office is a sign of genius. What does benefit creativity is
exposing one's mind to a variety of influences, sources and types of
information. A mind that can stay focused despite diverse stimulation, and
that can produce enough ideas so that some of them might be truly great,
must be an organized one.
Continued in article
Bob Jensen's threads on asynchronous versus synchronous learning ---
http://faculty.trinity.edu/rjensen/255wp.htm
Click here to return to the top of this document.