Appendix: Toward Assessment Worthy of the Liberal Arts

 

The Truth May Make You Free, but the Test May Keep You Imprisoned

 

by Grant Wiggins, Consultants On Learning, Assessment, And School Structure

 

    I confess that I have been kicking myself for getting involved with this topic. The more I looked at the title I obligated myself to, the more nervous I became. For one thing, I do not know as much as I should about current efforts to assess higher education. For another, I do not think anybody in his or her right mind can address this topic intelligently in so little time. And third, I think that the problem specified in the title of my talk is an insoluble one. It confronts us with one of many inescapable dilemmas about the liberal arts: the freedom of thought to go where it will versus the apparent need for uniformity in the testing process. In other words, we are not going to "solve" the assessment problem in the liberal arts, now or later. We are going to negotiate it—painfully; we are going to have to deal with some frequent, uneasy compromises.

    So what I intend to do is a bit more modest than perhaps it first seemed. My aim is not so much to lay out a complete vision, but to give you my sense of the subtle but profound shifts that would be required if we were going to be serious about assessing for a liberal arts education. Second, I am going to propose to you a set of principles that we might call upon when liberal education is jeopardized--as it always will be--by an overly utilitarian or vocational view of teaching and learning. I think of these principles as mere first cuts, but perhaps they can hold you in good stead on a rainy day. Third, I will offer what I hope are some provocative and useful illustrations of alternative forms of assessment that befit the liberal arts. Many of the examples happen to come from the K-12 arena but nonetheless apply to your situations as well. I would encourage you, therefore, to resist a common nasty, little habit. If I should make reference to a fifth grade teacher's example, try not to be snooty about it. It is harder than you think to resist the feeling, and harder still to develop the almost anthropological mindset that enables one to find insight into one's own teaching from very different places in the system.

 

Thoughtless Mastery

    Let us begin thinking about dilemmas in education by returning to the first known assessor in the liberal arts. I am thinking, of course, of Socrates, the Socrates of the dialogues of Plato, where we regularly see those who either appear to be or profess to be competent put to the "test" of question, answer, and--especially--sustained and engaged conversation. (The dialogues themselves, of course, are filled with dilemmas. Many of them are left unresolved: a reminder of how these arts are meant to lead to questions, not answers--the little burrs that get under your saddle.) Socrates the assessor: he is certainly a strange one. He does not seem to have nice answer keys or scoring rubrics by his side. Yet I think that there is something to learn from thinking about assessment from a Socratic point of view.

    I would like to view these issues through my favorite piece of literature, the dialogue called "Meno." Some of you no doubt know it. Meno, a brash young fellow, comes up to Socrates. The first line of the dialogue is, "Tell me, Socrates, how do we become virtuous?" In other words, he is asking how morality develops: through upbringing? moral education? by nature? Socrates responds in a very annoying and typically Socratic way. He says, "Well, I cannot answer that question. I do not even know what virtue is." Meno is clearly astonished to think that this could be possible, that a bona fide, certified sage does not know what everybody knows, namely, what it means to be good. But of course, after Meno makes the foolish mistake of venturing to tell Socrates what virtue is, Socrates proceeds to undress him two or three times.

    Finally, in exasperation, Meno says a terribly revealing thing that goes to the heart of the distinction between conventional assessment done well and an assessment for the liberal arts. Meno says, "Well now, my dear Socrates, you are just what I have always heard before I met you. Always puzzled yourself and puzzling everyone else. And you seem to me to be a regular wizard. You bewitch me. You drown me in puzzles. Really and truly my soul is numb. And what to answer you I do not know." And here is the important part. "Yet I have a thousand times made long speeches about virtue before many a large audience. And good speeches too, as I thought. But I have not a word to say at all as to what it is."

    Meno's comment (and indeed the progress of the whole dialogue) ironically reveals what so differentiates conventional academic mastery from excellence befitting the liberal artist. Meno is reduced to speechlessness, he thinks, because of the sophistry of Socrates' questions and analyses; the thoughtful reader knows, however, that Meno does not know what he is talking about. And yet Meno is a conventionally successful student. How do we know? Throughout the dialogue Meno is constantly dropping references--the ancient equivalent of footnotes--to all the famous people who say this and that about virtue, which he, of course, agrees with. And it is no doubt the case that Meno could be a successful speaker--effective, convincing. The point of the dialogue, of course, is that such rhetorical skill using borrowed ideas is not understanding; competent presentations are not sufficient. That is not what a liberal education is about.

     What Socrates wants us to see--what Plato want us to see by the way in which the dialogue is writtenwis that the conventional view of education is actually quite dangerous. If one gets better and better at what one does, one is less and less likely to question what one knows. Meno has been a dutiful student. (We are also meant to know that his name is a pun: It is very close in Greek to the word for memory: menon-mnemon.) Meno is an effective memorizer, able to make effective speeches with references to famous people. Isn't that what too much of our assessment is already about? Don't we too often fail to assess whether the student can do anything more than borrow quotes, facts, and figures?

     But we also know from history that the real Meno was a nasty fellow: clever, ruthless. We are meant to know that. Because there is ultimately a lesson to be learned about "control" over knowledge and the ends to which "mastery" is put. Liberal education can never co-exist happily with other, more "practical" views of education because a liberal education is about rooting out thoughtlessness--moral as well as intellectual thoughtlessness.

     There is, alas, such a thing as "thoughtless mastery" (as I have elsewhere termed it) and our syllabi and assessments tend unwittingly to reinforce it. Many of our students are quite good at this thoughtless mastery; you all know it. You know those looks in class, those mouth-half-open looks, the eyes slightly glazed; when people are fairly attentive but the brain does not seem to be quite engaged; when, alas, their eyes only focus to check scores on other people's papers, and to press you for extra points here and there.

      Paradoxically, many professions require unthinking mastery--and run the risk of an amoral technical approach to life. I think we forget this. I do not want the pilot who flew me to Washington to be questioning his knowledge or his existence. Nor do I want my brain surgeon to be thinking about what virtue is. One of my passions is baseball, and I was recently reading George Will's new book called Men At Work on the craft of playing and managing major-league baseball. There is an odd but insightful phrase in it about this kind of thoughtless mastery that rings quite true. The good hitters talk about not thinking too much--that it is very dangerous to do so. Rather, what has to take over the hitter is something called "muscle memory'--a wonderful phrase for the kind of unthinking skill that we admire in athletes.

     There is no reason, however, for colleges and universities to assume that their job is to promote unthinking mastery of others' ideas (while also abetting the other forms of thoughtlessness that too easily follow). Colleges are derelict, I think, in giving up the only sanctioned time when we have a moral obligation to disturb students intellectually. It is too easy nowadays, I think, to come to college and leave one's prejudices and deeper habits of mind and assumptions unexamined--and be left with the impression that assessment is merely another form of jumping through hoops or licensure in a technical trade.

     Certainly we say we would like to see more "real" thinkers, and we bemoan doltish behavior in our students. I think we protest too much. Our testing and grading habits give us away. If you do not believe me, look how often students give us back precisely what we said or they read. On the other hand, you should not think that I mean rigor does not matter. That is part of the dilemma. The great mistake that has been made in school reform by many so-called progressives, and by much of the alternative schools movement, is to assume that to be liberated is to be liberated from discipline. That is a mistake, and it is one reason why alternative school people end up shooting themselves in the foot: because they produce a lot of free spirits who are not always very capable. If I had to choose, I might go with the alternative schools, but it is a bad choice and it shows that we have not negotiated the dilemma in K-12.

    So we have to think about rigor. We have to think about alternative assessment as more than just engaging students better, which it invariably does (you know this if you have done simulations, case studies, portfolios, or dramatic presentations with your students). We need more than engaging activities. We need truly standard-setting and standard-revealing assessments. Or as psychologist Lauren Resnick puts it: What we assess is what we value. We get what we assess, and if we don't assess it, we won't get it. True about rigor, but also true about the intellectual virtues.

     Some of you know, if you have read some of the things that I have written on alternative assessment, that one of my definitions of authentic assessment is that it is "composed of tasks that we value." It is not a proxy. It is not an efficient system to shake out a grade. Efficiency and merely technical validity as the aims of assessment will undermine liberal education. Rather, the test should reveal something not only about the student but about the tasks and virtues at the heart of the subject--its standards. But it is damn hard to design tasks to meet those criteria. It is very easy to score for efficiency and to look at what is easy to score rather than what is essential.

 

Three Dilemmas

    Let me cite three other dilemmas before giving you some principles and examples of how we might think about assessment that would do justice to the liberal arts. The first dilemma, confronting you more often as a teacher the higher up you get in the system, is whether to stress students' mastery of the ideas of others or mastery over their own emerging ideas. In fact, we do believe that it is important for students first to control subject matter and to acquire skill within the discipline before they get "creative." Or to paraphrase Thomas Kuhn, one must have complete control over the existing "paradigm" if dramatic new paradigms or original thoughts are to occur.

    Whatever Kuhn's merits as a historian and philosopher of science, I think he is dead wrong about education. I think it is terribly important that would-be liberal artists immerse themselves, from the word go, in questioning the paradigm as they learn it: They should study it, poke it, prod it, and not wait until they have mastered it--because you can have a long wait. And many of your bright and able minds are likely to drop out mentally or physically because they cannot wait that long. Conversely, the ones that stick around may be more dutiful than thoughtful.

    Inevitably, if we first demand control over the subject matter in its entirety, we run a moral as well as an intellectual risk. We run the risk of letting the student believe that authority and authoritative answers matter more than inquiry. We may well end up convincing students that "Knowledge" is somehow something other than the result of personal inquiries built upon questions like theirs. And in fact, many students do believe that: There is "Knowledge" over here and never the twain shall meet.

     A second way to put the dilemma is more classic: useful versus useless knowledge. There is an important sense in which the liberal arts are useless, summed up in that little comment supposedly made by Euclid 2,000 years ago when someone complained that geometry was not good for very much. He said, well, give him three drachmas if he has to get some usefulness out of studying it. But there is a more important truth in this desire. It is not at all clear that this unending inquisitiveness and poking over, under, and around knowledge is useful. Indeed, I can tell you from working with adolescents for so many years (prone to outbursts of honesty and not feeling the need to appear like eager apprentices), that many of them regard it as profoundly useless. On the other hand, we must ourselves keep clear the distinction between "useful" (or "relevant") and "meaningful." Students are not entitled to usefulness in a liberal education, but they are entitled to a meaningful encounter with essential ideas. We often disappoint--either by pursuing ideas that are too relevant but transitory, or by being insensitive to their need for provocations, not packages of pre-digested "knowledge," to chew on.

     Third, we have to recognize that the urge to shun the liberal arts may have a great deal to do with the essential urge to feel competent. People go to school, it seems to me, indirectly to feel good about themselves. They want to develop competence because they want to develop confidence--or is it the reverse? The trouble with a liberal education is that it does not satisfy that need at all. It is unpleasant. It is disturbing. Many people drop out mentally and become hyper-competent because they cannot deal with the ambiguity and uncertainty that is the hallmark of the liberal arts.

Well, then, suppose I am right about this. Suppose we are in danger of treating assessment in higher education--as we are now increasingly in danger of treating assessment in lower education--as certification that a student possesses sanctioned knowledge. Where would we look for effective alternative strategies? How can we highlight the liberal arts side of the dilemma? What principles might guide us in designing assessments for the liberal artist in training?

Let me offer you ten tentative principles.

 

Principle 1

    The heart of the liberal enterprise is not a mastery of orthodoxy but learning to justify one's opinions. Because the modern university has its roots in the Middle Ages and in religious training, it is built upon an irresolvable tension between orthodoxy and the promotion of inquiry. We tend to forget that. To this day, it seems to me, we still lean pretty heavily on the orthodoxy side: Up until the graduate experience, students have first to demonstrate their control over other people's knowledge. Yet we would be wise to begin our reforms from the perspective of the ultimate educational experience with which we are all familiar: the dissertation and oral in defense of a thesis. We should think of all assessment as designed primarily to give students an opportunity to justify opinions that are being developed as they explore subject matter.

    This implies that one of the most important things that we can do in assessment is to examine the students' response to our follow-up questions and probes of their ideas. It implies, for instance, in assigning a paper and evaluating it, that the student should have to formulate a response, to which we then respond as part of the formal assessment process, not as a voluntary exercise after the test is over or the paper done.

    Taken to the limit, I would argue that one of the most important things that we can do with students is to assess them on their ability to punch holes in our own presentations. They have a right to demand from us justification of our point of view. That is what the liberal spirit is about. It sends a moral message that we are both, student and teacher, subservient to rational principle.

 

Principle 2

    The second principle is that we really need to think of the student as an apprentice to the liberal arts. And like all apprentices, students should be required to recognize and produce quality work. They should not get out of our clutches until they have produced some genuinely high-quality work. Now, what do I mean by that? Well, it is really a subtle shift in thinking. We all expect quality as teachers, but I do not believe that we demand it.

    For instance--and here is one of those sixth grade examples--there is a teacher in Louisville who in one of her first assignments to her social studies students demands that every student read a book and do a book report. Not a particularly interesting task, but what is fascinating is what she demands. She demands that the paper be perfect. She demands that the students not turn it in until it is. She demands that they seek out anyone and everyone who will help them make it perfect.

    Well, needless to say, the kids freak out. Especially the bad ones who are convinced that they cannot produce quality work. To make a long story short, they do. Oh, we could quibble with the idea of a perfect paper, but the kids understand full well what is meant. They really do. It is quite something to see. They understand that they have to ratchet up the seriousness with which they work. That they cannot wait to find out the quality of the work they produced. That they have to produce the quality work first. Making a point that many of you know is now critical to the alternative assessment conception: Assessment and self-assessment must be intertwined if we are serious about empowering people. To demand quality is also to structure assessment so that the student does not merely have the opportunity to rehearse, revise, rethink, but is actually required and expected to do so.

    One of my favorite assignments when I taught at Brown was to ask students for their final paper to rewrite their first paper, based on all they had since learned or thought. A number of the upper-classmen told me that it was the most important event in their four years at Brown. They were astonished to see how their thinking had changed. They were astonished to discover how sloppy that early work seemed to them in retrospect. In short, they were learning about quality.

    Further, they were learning about thinking, that thinking does not stand still and should not. Demanding quality, in other words--and this is part of the shift in thinking that is required--means we begin to focus our assessment on what Aristotle called the intellectual virtues. Does the student display craftsmanship, perseverance, tolerance of ambiguity, empathy when everyone else is critical, a critical stance when everyone else is empathetic? Can the student, without prodding, re-think and revise a paper or point of view? A liberal arts education is ultimately about those intellectual virtues. When all of the knowledge has faded away, when all of the cramming has been forgotten, if those intellectual dispositions do not remain, we have failed.

     Now, some people get very squeamish about assessing things like perseverance, style, craftsmanship, love of precision. I do not. If we value it, we should assess it. That does not mean that we are arbitrary. That does not mean that we are subjective. Yet, we have to worry about validity and reliability. In fact, what I think it means to assess habits of mind is not to directly score them at all. But rather to devise tasks that require the habits we value.

     My metaphor for this is "Outward Bound." Assessment should be like intellectual Outward Bound. It should reveal to the student what we value as traits in them by the virtues required to accomplish the task at hand. It should not be possible to do an end-run around those habits; students who can get A's by missing class and cramming are telling you something about the failings of your assessment system.

     Sometimes it is as subtle a shift as sending the message day in and day out that quality matters and you are held accountable for quality. One of my favorite little tricks in that regard comes from Uti Treisman at Berkeley and his work with minority mathematics students. He demands that every piece of work the students hand in be initialed by another student; students get a grade both for their own paper and for the paper on which they sign off. This sends a message loud and clear that quality matters, that you are personally responsible for quality, and that it is in your interest to find out about quality before hearing from the authority. Quality control is about avoiding poor performance before it happens.

 

Principle 3

    This leads directly to Principle 3, a point familiar to many of you who have been at this kind of work, but one that cannot be made often enough. A liberal arts assessment system has to be based on known, clear, public, non-arbitrary standards, and criteria. There is no conceivable way for the student to be empowered and to become a masterful liberal artist if the criteria and standards are not known in advance. The student is kept fundamentally off-balance, intellectually and morally, if the professor has a secret test and secret scoring criteria.

    Consider the performance world, as opposed to the academic world, and how much easier it is for performers to be successful because of this very basic fact. The test is known from day one. The music, the script, the rules of debate, the rules of the game are known: genuine mastery in the performance arena means internalizing public criteria and standards until they become one's own. Unfortunately, in education, especially in higher education, there is a vestige of our medieval past, when tests were a bit of mystery and novices had to divine things. I was disappointed to learn when I was a teaching assistant at Harvard that most undergraduates are still not allowed to see their blue books. And then somebody told me that at Oxford and Cambridge they burn them.

    I think this is an unfortunate and deadly tradition. It is also a legacy of tests used as gatekeepers, not as equitable vehicles designed for displaying all that a student knows. Most people in this room, I suspect, would say it is the student's responsibility to figure things out, to respond to the test as the test demands, and to produce quality work on our terms. I am not convinced of that. Why isn't the university required to meet students half-way and give them a chance to reveal their strengths and play from their strengths? It would be as simple as giving people the option of alternative forms of doing the same assignment.

    But I think it runs much deeper. We are still using testing as a sorting and categorizing system. And elitism should not be confused with meritocracy. Our most common habit in scoring and grading, namely scoring on a curve, is unjustifiable in my view. Its sole purpose is to exaggerate difference rather than reveal strength. It makes our life easier and it relieves us of justifying the grades and scores that we give. It is needlessly debilitating--as opposed to a challenge that we can rise to when we know, understand, and appreciate the criteria.

    Of course, many of you know the solution. Scoring rubrics, model papers, video-taped model performances, anything that can give students an insight into, allow them to enter the field and acquire its standards, by seeing exemplary performance before they do their work. I do not know why in the world we keep such matters a secret. It is cuckoo--and dysfunctional.

 

Principle 4

    It follows that what a liberal education is about--and what assessment must be about-is learning the standards of rational inquiry and knowledge production. And this implies that self-assessment is a critical and early part of assessment. Now, many of you know about Alverno's use of self-assessment and it has been borrowed by many of us. I just want to give you one of my favorite Alverno examples because I think it illustrates so well different ways of thinking about this.

    One of Alverno's goals for students is competency in oral communication. Early on, a student must give a video-taped talk and so one's first hunch is, oh, well, you are going to assess the talk. No. After the student gives the first talk and it is video-taped, the student is assessed on the accuracy of the self-assessment of the video-taped talk. That is a fundamental shift in point of view. If we want people to gain control of important habits of mind and standards, then they have to know first of all how to view those things accurately and apply criteria to their own work, and not always depend upon another person to do that.

    It is also a basic lesson in habit development. You have to know what you are supposed to be doing before you can do it. And that knowledge is crucial in making you stick with it and believing that it is possible. Otherwise I do not think any of us would quit smoking or lose weight. It suggests as a practical corollary that no major piece of work should get turned in without some self-criticism attached to it. And that self-criticism should be assessed for its accuracy.

 

Principle 5

    Most education, it seems to me, treats the student as a would-be "learned spectator" rather than a would-be "intellectual performer." The student must metaphorically sit in the bleachers while others, mostly professors and writers of textbooks, perform. This arrangement takes us back to the idea that competency involves just remembering and applying what others say. It has dangerous consequences because it induces intellectual passivity. In an education for a would-be performer the student would experience the same "tests" that face the expert in the field--having to find and clarify problems, conduct research, justify one's opinion in some public setting--all while using (other people's) knowledge in the service of one's own opinion.

    Let me give you a couple of my favorite examples of this. One of the finest classes that I have ever observed at any level was at a high school in Portland, Maine, where a veteran teacher offered a Russian history course. The entire syllabus consisted of a series of chronological biographies. It was, however, the student's job to become each person, in turn, in two senses: through a ten-minute talk, and then a simulation. After four or five students had presented their talks (and been assessed by other students on their talks), they had a Steve Allen "Meeting of the Minds" press conference which was chaired by the teacher; the "journalists" were the other students. Each party scored the other for its performance.

    Now, I do not know about you, but I have sat through a lot of dreary reports. These were not dreary. In fact, they were as interesting and informative as any reports I had ever heard. I went up to the teacher and said, golly, how did you get them to do that? He said, well, it was very simple. There were only two criteria by which they were going to be judged and they were (a) whether the talk was accurate, and (b) whether it was interesting. This was real performing and using knowledge.

 

Principle 6

    This one follows from Principle 5. A liberal artist, if he or she has "made it," is somebody who has a style. Somebody whose intellectual "voice" is natural and clearly theirs. Read the turgid prose that we receive and you know that we are failing to develop style, voice, and point of view. (Read our own writing in journals ... ) Students are convinced we want the party line, and that the quality and insight possible in compelling prose is not necessary. It is an option.

    There would be a simple way to get at this. After writing a lengthy research paper with all the requisite footnotes and bibliographical information, the student could be asked to turn the paper into a one-page piece to be delivered, in an engaging and insightful way, to an audience of lay-persons. But it is not just an aesthetic issue, this business of style. It is a question of one's inner voice. One's intuition. The seed of a new idea that is easily crushed if it is not allowed to be heard. All of these are related to the idea of conscience, and, of course, it is no coincidence that Socrates talked about his little voice.

    It is easy for students in American universities to lose that little voice. But that little voice is not just a "personal" voice irrelevant to "academic" accomplishment. It is the voice of common sense. It is the voice that can turn around and question the importance of what one has just spent two months working on. It is the little voice that says, ahh, come on, is this really that important? Or it is the little voice that says, you know, there is probably another way to look at this. It is the little voice that says, I have a feeling that there is something behind what the professor is saying, but I do not know enough to really pursue it so I will not. It is the little voice that most of us do not hear in our students unless we ask for it. An assessment should ask for it.

    Such assessing need not be difficult. I saw an English teacher do it. In using peer editing, he told his students that they should reject and turn back any paper that was boring or slapdash--and mark the exact spot in the paper where they began to lose interest. Nothing sends a message faster to students about writing and its purpose and quality. Nothing sends a message quicker that technical compliance with criteria is not always of primary importance.

    There is another point to be made about voice and style. The thing that is so ghastly about academic prose is that one really does sense that it is not meant for an audience. And, of course, sometimes it is not. It seems to me that if we are serious about empowering students, we must get them to worry about audience in a deeper way. We must demand that their work be effective. We must demand that it actually reach the audience and accomplish its intended purpose. There is nothing more foolish, in my view, than saying, "Write a persuasive essay" without the students having to persuade anybody of anything. So let us set up situations in which the student has to persuade readers, or at least get judged by an audience on more than just accuracy. Even Socrates knew, in the clash of Reason and Rhetoric, that teaching had to be not merely truthful but effective.

 

Principle 7

    Too often in assessment we worry about whether students have learned what we taught. This is sensible, of course. But let me take an unorthodox position. Such a view of assessment, taken to extremes, is incompatible with the liberal arts. One important purpose of those "arts that would make us free" is to enable us to criticize sanctioned ideas, not merely re-tell what was taught.

    A less confrontational way to make the point is to remind ourselves that it is the astute questioner, not the technically correct answerer, who symbolizes the liberal artist. The philosopher Gadamer (with an explicit homage to our friend Socrates) argued that it is the dominant opinion that threatens thinking, not ignorance. Ensuring that students have the capacity to keep questions alive in the face of peer pressure, conventional wisdom, and the habit of their own convictions is what the liberal arts must always be about.

    Admittedly, some knowledge is required to ask good questions and pursue the answers we receive. But if we are honest about this we will admit that the kind of exhaustive expertise we typically expect in students is over-kill. After all, children are wonderful and persistent questioners; recall the wisdom of H.C. Andersen's The Emperor's New Clothes. Indeed, academics are invariably prone to making the mistake Gilbert Ryle called the Cartesian fallacy: assuming that "knowing that" must always precede and serve as a condition for "knowing how." No person who creates knowledge or uses knowledge to put bread on the table would ever be guilty of this fallacy. All apprentices or would-be performers learn on the job, yet as teachers we tend to over-teach or "front load" knowledge. So a good pedagogical rule of thumb would be: teach the minimum necessary to get the students asking questions that will lead to your more subtle goals.

    We would do well, then, to think of our task as introducing the student to cycles of question-answer-question and not just question-answer--with one aim of a course being to make the student rather than the professor the ultimate initiator of the cycle. To postpone developing students' ability to ask important questions in the name of "mastery" is to jeopardize their intellect. Good judgment and aggressive thinking will atrophy if they must be endlessly postponed while professors profess. In any event, the most important "performance" in the liberal arts is to initiate and sustain good question asking.

    A very mundane point about testing can be made out of this esoteric argument. We rarely assess students on their ability to ask good questions. Indeed, we rarely teach them a repertoire of question-asking strategies for investigating essential ideas and issues. It should become obvious to students through the demands of the course and our assessment strategies that question-asking is central. Too often, however, our assessments send the message that mastery of the "given" is the exclusive aim, and that question-asking is not a masterable skill but a spontaneous urge.

 

Principle 8

    This principle follows from Principle 7. The aim of the liberal arts is to explore limits-the boundaries of ideas, theories, and systems. To paint the starkest picture of the difference between a "liberal" and a "non-liberal" view of the disciplines, therefore, we might see our task as teaching and assessing the ability to gauge the strengths and weaknesses of every major notion we teach--be it a theorem in mathematics, a hypothesis in science, or literary theory in English. We need to know whether students can see the strengths and weaknesses of "paradigms." This would include the limits of a theory not only within a subject but across disciplines, as when we apply the rules of physical science to the human sciences.

    There is no novelty in this idea. I am invoking a notion about the liberal arts developed thirty and more years ago by Joseph Schwab at Chicago. He termed such a view of education the art of "eclectic," and I encourage you to return to his essays for numerous suggestions on how to help students explore the merits of sanctioned truths.

    I fear that we no longer know how to teach science as a liberal art in this sense. When we make science merely abstruse and technical, we make it increasingly unlikely that nonscientists will profit from studying science enough to support intelligent science policy as adults. And we encourage science students to become too technical and insufficiently critical. I really think that the first years of study in college (and certainly throughout secondary school) have less to do with "mastering" science and more to do with orthodox algorithms-learning metaphysics instead of physics, sanctioned truth vs. the unstable results yielded by methods and questions that transcend the current results.

    I know this weakness in our science students first-hand from my high school teaching days. My students did not understand, for example, that error is inherent in science and not merely the fault of immature students or poor equipment. (Many believe that when the "big boys and girls" do their measuring, the results are exact.) Nor did many of them realize that words like gravity or atom do not correspond to visible "things" to be seen directly.

    The point can be made another way. We still do a poor job of teaching and assessing the student's grasp of the history of important ideas. I know of no other method by which inappropriately sacred truths can be more effectively demystified. What questions was Newton, then Einstein, trying to answer? What did the first drafts of a history text look like, and why were they revised? To ask questions like these is to open up a new and exciting world for students. To be smug about our knowledge and to distance ourselves from "crude" and out-dated theory is to ensure that we repeat the mistakes of our smug and parochial elders.

    Consider the history of geometry, the very idea of which strikes many people as an oxymoron. Many college students are utterly unaware of the problems that forced Euclid to develop an awkward parallel postulate (which was instantly decried by his colleagues). So much for self-evident truths, that glib line found in superficial views of Greek mathematics! Further, most students are unaware that non-Euclidean geometries can be proven to be as logically sound as Euclid's; they have no idea how that result transformed epistemology for all fields of study.

    The consequence of our failure to reveal to students the history of important ideas is two-fold. For one, students easily end up assuming that axioms, laws, postulates, theories, and systems are immutable, even though common sense and history say otherwise. The second result follows from the first and does lasting harm to intellectual courage in all but our feistiest students. Students never grasp that "knowledge" is the product of "thinking"—thinking that was as lively, unfinished, and sometimes as inchoate as their own. One major reason for the intellectual poverty in this country is that most students either become convinced they are incapable of being intellectual, or they are uninterested in it if it involves only arcane expertise in a narrowly framed subject.

    Some practical implications for assessment? First, we should require students to keep notebooks of reflections on coursework, their increasing knowledge, and important changes of mind about that knowledge. Second, we should assess this work as part of the grade. I did so for many years and found it to be the most important and revealing aspect of the students' work. I also learned a lot about how their thinking evolved in a way that improved the courses I taught. Third, even the most technical training should ask students to do critical research into the origins of the ideas being learned so that students can gain greater perspective on their work. If we fail to do this, whether out of habit or rationalization that there is no time for such reflective work, we risk producing a batch of thoughtless students.

 

Principle 9

    Principle 9 extends the moral implications of Principle 8. When we encourage narrow, unchecked expertise, we may unwittingly induce students to be dishonest about their ignorance.

    I am not even talking about the more heinous crime of cheating, something we know is all too common. Rather, I am talking about the moral obligation of the liberal artist to emulate Socrates' trademark: his cheerful admission of ignorance. Alas, our students rarely admit their ignorance. One of our primary tasks should be to elicit the admission and not penalize it. But the student's willingness to take such a risk depends upon our doing so. It is then, as the "Meno" reminds us, that mutual inquiry and dialogue become possible because we are placed on equal moral footing as thinkers. More pointedly, our inclination to "profess" is always in danger of closing off the doors through which our students can enter the liberal conversation without excessive self-deprecation. So many of our students preface a wonderful idea by saying, "I know this sounds stupid, but ..."

    Let our assessments, therefore, routinely encourage students to distinguish between what they do and do not know with conviction. Let us design scoring systems for papers that heavily penalize mere slickness and feigned control over a complex subject, and greatly reward honest admissions of ignorance or confusion. Or, let us go the next step and ask students to write a second paper in which they criticize the first one.

 

Principle 10

 

    The tenth and last Principle extends the point. Intellectual honesty is just one aspect of self-knowledge and the absence of self-deception. One of my favorite notions was put forward by Leo Slizard in talking about how to assess doctoral candidates. He argued that students should be assessed on how precisely and well they knew their strengths and limitations--and that it was a mistake to err greatly in either direction.

    I am not arguing for professors to become counselors. I am arguing for them to improve students' ability to self-assess, and to make sure that accurate self-assessment is more than a pleasant exercise. It is an essential tool for ensuring that students have neither excessive nor insufficient pride in their work, either of which closes off further intellectual challenges and rewards.

    The inherent danger of scholarship is not so much error as blind spots in our knowledge-hidden by the increasingly narrowed focus of our work and the isolation that can then breed worse arrogance. Excessive pride leads us not only to ignore or paper over our doubts but more subtly to be deceived about the uniqueness and worth of our ideas--we forget that it was a conversation in the coffee shop or reading an article that sparked the idea. A few collaborative assessment tasks, with some reflection on the role of each contributor, would provide useful perspective for everyone.

    It follows that we should assess class discussions more than we do. We again fail to assess what we value when we make it possible for students to learn everything from only listening to us and doing the reading. I and many others have developed some good material for assessment (and self-assessment) of class discussions, and I encourage you to develop some methods of your own.

    Which of course brings us back to Socrates. What the casual reader of Plato always fails to grasp--including some overly analytic philosophers, I might add--is that the dialogues invariable are about character, not "theories" of virtue, knowledge, or piety. The twists and turns of dialogue, the sparring with Sophists or young know-it-ails--ultimately all this is meant to show that character flaws, not cognitive defects, impede the quest for knowledge. It is one's attitude toward knowledge that ultimately determines whether one will be a liberal artist or merely proficient.

    As Socrates repeatedly reminds us, we must love wisdom so much that we question our knowledge, even our pet ideas if need be. By extension, the more we gain confidence in our ideas, the more we must become vigilant about finding knowledge in unexpected places--and let others who seem incapable of it teach us something, as our students often do.

    It is not a canon--of ideas or books--that defines the liberal arts, but a set of very hard-won virtues. Like all sophisticated dispositions, these liberal habits are typically only revealed when they are challenged. It is only when peer pressure is greatest, be it in the classroom with students or at conferences with our peers, that we learn who has the power to keep questions alive. The liberal arts, properly speaking, do not make you free; they keep you free. Wisdom--Socrates knew--reveals itself when persistent inquiry is threatened: externally by custom and "oh, everyone knows ... ;" and internally by the tendency to rationalize our own habits, beliefs, and fears.

    How much do students really love to learn, to persist, to passionately attack a problem or task? How willing are they, like the great Indian potters of New Mexico, to watch many of their half-baked ideas explode, and start anew? How willing are they to go beyond being merely dutiful, perfunctory, or long-winded? Let us assess such things, just as the good coach does when he or she benches the talented player who "dogs" it.

    We are then quite properly assessing not skill but intellectual character. It is to our detriment and the detriment of the liberal arts if we feel squeamish about saying and doing so. Let us "test" students in the same way that the mountain "tests" the climber--through challenges designed to evoke whether the proper virtues are present. And if not present, the quality of the resultant work should seem so inadequate to the student that little need be said in the way of "feedback."

    Let our assessments be built upon the distinction between wisdom and knowledge, then. Too subjective? Unfair? Not to those who have the master's eyes, ears, and sense of smell-tact, in the old and unfortunately lost sense of that word. For these traits are as tangible as any fact, and more important to the student's welfare in the long run. It is not the student's errors that matter but the student's response to error; it is not "thoroughness" in a novice's work that reveals understanding but awareness of the dilemmas, compromises, and uncertainties under the arguments one is willing to stand on.

    If our testing encourages smug or thoughtless mastery--and it does--we undermine the liberal arts. If our assessment systems induce timidity, cockiness, or crass calculations about grades and the relevance of today's assignment, we undermine the liberal arts. If our assessments value correctness more than insight and honesty, we undermine the liberal arts. If our assessments value ease of scoring more than revealing to students the errors or tasks that matter most, we undermine the liberal arts. Let us ensure, above all else, that our tests do just what Socrates' tests were meant to do: help us--and our students--to distinguish the genuine from the sham authority, the sophists from the wise. Then we will have assessments that are worthy of our aims.

 

Back to Getting Started with Assessment