Volume XXXI, No. 29 | December 4, 2009
The Demand to Measure: Destroying the Bloom of Learning
Perhaps the most dominant force in education today is the growing demand for “accountability.” Teachers and administrators are commanded to measure educational successes and failures. But what do we measure? And how do we measure it?
What we measure can be categorized using Bloom’s (1956, revised 2001) taxonomy of Learning for Mastery. Speaking roughly, primary- and secondary-level educational strategies are about remembering and understanding; college-level work is applying, analyzing, and evaluating; graduate-level program work is evaluating and creating. These are the domains of accountability for learning. But how are they to be measured?
To what measures are educational objectives subject? Remembering is the objective that is uniformly subject to a ratio measure. Count the number of things that are to be remembered, count how many the student reproduces upon demand, and calculate the percentage. With a very few, very narrow exceptions, every other level in Bloom’s taxonomy is subject to nothing beyond ordinal (relational property of things) or nominal (rule-based assignment of numerals) measures. Furthermore, as one moves up the Bloom pyramid, variation among ordinal scores increases because the subjective element and/or the degree of skill and judgment required to assign a rank increases. Judges may find easy agreement as to relative quality within a set of paraphrases. But, as the bevy of famous letters from publishers who rejected what are now widely regarded as literary masterpieces attest, great difficulty arises in assigning rank to creative work.
Many in educational assessment fail to understand that “measure” and “ratio measure” are not synonymous, and they have a compelling—though intellectually dishonest and misleading—proclivity to treat any numerical assignment as if it were a ratio measure.
Ordinary letter grades make the point. At best, letter grades provide a rather crude ordinal ranking. Yet such grades are almost universally converted into a numerical “measure” from which “averages” are computed to three or four decimal places. Some persons place great importance on these results. But anyone who believes that these manipulations produce something of great worth demonstrates nothing but a thoroughgoing general ignorance concerning both measurement and significant figures. Such ignorance has profound consequences for self-deception and dumbing down of curricula.
These consequences are interrelated. In order to report ratio measures, teachers must dumb-down learning objectives to the lowest level of Bloom’s pyramid—remembering. The self-deception occurs when they dumb down without realizing it. I offer the following explanation.
One way to introduce philosophy at the college level is through classic source documents: readings from the likes of Plato, Descartes, Hume, and Kant. As I recall my undergraduate days (decades ago), every philosophy course I took used a simple anthology in which, at most, a one-page introduction prefaced each article. Our job as students was to find philosophical arguments in these sources, recast them in our own words, and evaluate the merits or lack thereof.
Today, I see few such simple anthologies. They have been replaced largely by colorful textbooks that present a limited number of selected passages from classic sources. These books then recast the passages in varied and multiple ways, present a sentence-by-sentence analysis, and provide a thorough evaluation of the strengths and weaknesses of each. As one of my colleagues pointed out, students do much better on tests when they study from these textbooks, as compared with working from source documents.
No doubt using these textbooks produces higher scores on bubble tests. But are the students learning higher-order thinking skills? Here, I fear that that my colleague may suffer from self-deception. Test questions may look exactly like the sort of questions that would focus on a student’s understanding, application, analysis, evaluation, or even creation; but that does not mean that the test either requires or measures a student’s understanding, application, analysis, evaluation, or creation. When “learning resources” do all the higher-order work, a student need only remember and reiterate.
No doubt the intent of textbook authors is to promote higher thinking. No doubt they do more and more of the students’ work because “objective” tests show higher scores when they do. But these improvements in no way measure higher learning. Rather, higher scores may be an indicator of how much the curriculum has been dumbed down to the lowest level—simply remembering.
Aristotle did not have Bloom’s pyramid, but Aristotle understood that the most valuable things cannot be taught. They must be learned through trial and practice. Students can improve their understanding of higher-learning domains by watching masters at work. But to remember what the masters did is not to become a master. Students must themselves analyze in order to learn analysis. Merely being able to report on someone else’s analysis does not achieve proficiency in analysis. Of course, the learner’s first efforts at analysis will be crude. That is a simple fact about learning. Teachers can coach and inspire, but they can in no way eliminate the necessity that students do analysis for themselves. The same holds true for all domains of higher learning.
There is a great irony here. As the drive to measure outcomes has grown, test scores have increased. But so has the general perception that graduates are less and less well prepared to function on the job and in society. Could it be that the two increases are related? Of course, we must assess student work. But if we really care about higher learning, we must understand that that assessment must be ordinal. We must understand the strict limits to which ordinal measures are subject. We must educate about the limits of measurement. We must fight the demands to report precision beyond what nature allows. And we must take care not to deceive ourselves into lowering the domain in the pretense of raising standards. To do any less endangers higher learning and diminishes us all.
Charles E. Cardwell, Associate Professor, Philosophy
For further information, contact the author at Pellissippi State Community College, P.O. Box 22990, Knoxville, TN 37933-0990. Email author.
