A Guide to Understanding Learning Outcomes

Tuesday, August 14, 2012

By Bruce Fraser
 
Introduction
 
The recent attention to accountability and transparency in higher education has inspired a new nomenclature to describe certain aspects of instruction. This language is unfamiliar to many, even if the foundational practices it is meant to describe are intrinsic to traditional pedagogy. "Learning outcomes,” "assessment,” "data-driven instruction” and similar terms now dominate conversations about education; as a result, it is important that those in the field are fluent in the new terminology and can translate well-established, historically justified practice into this nomenclature.
 
This short guide is meant to clarify the terms and relationships of this new language and to codify the concepts and methods of the rational process that is the foundation of good teaching. It is not meant to reduce what and how we teach to a set of procedures, or to force the complexity of human communication into a neat and tidy formula. On the contrary, much of what we teach and how we communicate defies easy analysis or reduction, and the artistic dimension of pedagogy must be respected and preserved. However, we cannot and should not avoid the responsibility to clarify and improve instructional methods where we can, and the learning outcomes discussion is, at root, just such an effort.
 
The material is presented Socratically as a series of questions and answers in an effort to make the presentation less cumbersome. The sequence of questions is meant to move the conversation from the level of a mere clarification of terms to a discussion of some of the more difficult philosophical and pragmatic issues associated with pinning down a complex and often messy process.
 
It is my hope that working through this material will reveal how the new emphasis on learning outcomes can be used as a catalyst for an important, faculty-driven conversation that is necessary for the health of an intellectual community at the heart of academe. Education remains the foundation of democracy, and the interest in improving instruction should be recognized as having broad implications not only for the economic future of our country but for the political health of our communities.
 
In short, the purpose of this work is to provide a conceptual framework for discussing the goals of teaching and the connections between subject areas, institutional responsibilities and the broader culture. The value of this framework lies not only in its ability to help educators translate between the traditional language of instruction and that of productivity and accountability (i.e., the language of business), but in providing a foundation for evaluating and improving what we do in academe– on educators’ own terms. I am convinced that a rational evaluation of what we do in the classroom is intrinsically desirable and serves the interests of those who want to preserve the integrity of scholarship and pedagogy in an age of accountability. Indeed, accountability is an obvious byproduct of scholarship, since the very concept of academic research is based on the rational justification of ideas through evidence and documentation. Teaching is similar, based as it is on a rational communication process, and the language of learning outcomes and assessment is one way of revealing the value of what we teach and how we teach it.
 
What is a Learning Outcome?
 
A learning outcome is any consequence of the learning process measured in terms of a change in the learner’s knowledge or skill level (assessed in terms of changes in behavior, since we don’t have access to what’s going on inside a person’s head by any other means).
 
Discussions about learning outcomes presuppose that only the intended outcomes are worth considering, and that such outcomes are the result of the instructional process. In reality, learning outcomes can be intentional or unintentional, and whether or not they are achieved depends not only on instruction but on the ability level of the student (the result of genetic endowment and the experiential history of the student) and a host of external factors. For educational purposes, the core idea is that how and what one teaches has a bearing on the end product of instruction, and that assessing learning outcomes gives us the data necessary for improving the quality of instructional methods and, ultimately, increasing the likelihood of student success.
 
What is a General Education Learning Outcome (GELO)?
 
The term "General Education” refers to a broad range of intellectual competencies associated with success in the complex social, economic, and political environments in which people live and work. The term ‘success’ is problematic, of course, since what counts as success is highly variable depending on what aspect of human interests one is discussing. Still, success can be fairly associated with such attributes as the ability to adapt to changing circumstances, to communicate well with others, to solve a range of complex problems, and to participate meaningfully in public life. Put in these terms, the concept of general education is not vacuous, even if it is difficult to pin down.
 
The term ‘General Education Learning Outcome’ refers to the identified range of competencies–whether intellectual or emotional–associated with the adequate preparation for life outside the classroom. The connection between these outcomes and what is sometimes referred to as the "real world” is intrinsic: without this connection, outcomes are provincial and academic, of interest only within the intellectual traditions of academia. If one fails to connect GELOs with the broader social context, these outcomes are easily conflated with specific, subject-area outcomes.
 
What is a Subject-Area (i.e., Discipline-Specific) Outcome?
 
In any field of study, there are learning outcomes specific to that field and that must be achieved if the student is to be credited with knowing the subject. For example, in the study of ancient history within the western tradition, it is essential to understand the role that 5th century Athens played in shaping the intellectual, political, and military history that followed. A student of ancient history must know who Themistocles was, what his role was in shaping Athenian political life, and so on. Yet this outcome does not rise to the level of a GELO because it is not indicative, taken by itself, of the broad competencies associated with a solid general education (although it is often the case that someone who knows who Themistocles was also has a variety of other, important competencies).
 
A subject-area outcome, then, is a learning outcome that is closely linked to the content of an area of study and is not by itself sufficient to establish mastery of the more adaptive competencies associated with proficiency in community life.
 
Are general education outcomes related to subject-area outcomes?
 
Yes, and in complex and interesting ways. Although the subject-area outcomes are too specific to qualify as GELOs, the acquisition of the Gen Ed outcomes depends on a student’s gradual mastery of the subject-area outcomes (see Appendix I).
 
This is not to say that one develops the broad set of competencies associated with general education by compiling more and more facts. In the case of the general education competencies, the whole is more than the sum of the parts. Through the ongoing engagement with subject-area content, the learner gradually extracts the principles and methods of thought that allow for adaptive, critical and creative thought.
 
The mechanisms behind this process are not well understood, but there can be no doubt that a good deal of this process involves inductive inferences such as generalizing from specific examples, the use of analogies and metaphors, and various forms of pattern recognition. Whatever the combination of mechanisms, and whatever the biological foundations of these mechanisms, the intellectual processes that take us from subject-specific outcomes to general education outcomes is at the heart of the learning process.
 
There are two other points that need to be made here. First, the development of general education competencies requires intellectual flexibility and a level of abstraction typical of higher-order thinking. The evolution toward good thinking requires that one becomes less dependent upon concrete examples and develops a facility with concepts. Without this transition, the learner would be incapable of applying ideas and principles in different contexts and novel situations.
 
Second, the increased proficiency with the Gen Ed competencies is evident through this interaction with subject area content, and hence can be assessed at the level of the subject-area outcomes. [1] For instance, a good critical thinker will engage content in history differently from a person who is incapable of analyzing the meaning of events, developing and testing hypotheses, and identifying causal connections. To the extent that the student demonstrates these higher-order cognitive abilities in relation to the study of history, critical thinking skills can be assessed within the subject area under investigation. In short, we see the results of good thinking at every level, and these results are measurable.
 
How does instruction relate to the mastery of learning outcomes?
 
This is the central question of education. That instruction helps guide a student to some level of mastery of the core instructional outcomes is a presupposition of all teaching, yet there remains considerable disagreement as to the most effective methods for promoting learning. Moreover, the question of whether peer-to-peer instruction, Socratic questioning, the use of various forms of media, self-study, or some other technique is best for promoting a student’s mastery of learning outcomes is complicated by the diversity of ability levels and interest both among the student population and among instructors. Instructors have different styles and students have different needs; navigating students through course content often involves a series of negotiations on both sides.
 
Nevertheless, the key to determining the effectiveness of different strategies rests on our ability to assess the degree to which our learning outcomes are achieved. Assessment is the empirical touchstone of teaching; without it, pedagogy becomes ideological and dogmatic. The identification of learning outcomes combined with the use of appropriate assessment measures is the hallmark of rational pedagogy. Good teaching is at its core a rational process.
 
What does it mean to say that teaching is a rational process?
 
In its most basic form, a rational process is an evidence-based, reflective, and open-ended approach to investigating an issue or solving a problem. Such processes vary in detail and application, ranging from the deductive proof-procedures of mathematics to the scientific study of the natural world.
 
To say that teaching is a rational process is to say that it must be (a) responsive to evidence bearing on questions of how people learn, (b) free of dogmatism (which is not to say free of theory), and (c) directed toward the socially significant goals and values that govern community life.
 
Parts (a) and (b) are essential to any rational pursuit. Part (c) is an affirmation of the social importance of education – its connections to the activities and purposes of the community (for further clarification of this point, see below).
 
Does the emphasis on learning outcomes endanger academic freedom?
 
Not necessarily, and certainly not in principle. Any serious conversation about academic freedom presupposes several key claims. First, academic freedom is constrained by the responsibility to teach a subject to the best of one’s ability; no one is free to be incompetent and use academic freedom as protection. Second, as all teaching takes it as axiomatic that there is a body of knowledge to be taught and that knowledge, by definition, is objective and verifiable, academic freedom cannot be used to support the idea that teaching is purely subjective, intuitive, and personal. In other words, academic freedom must not be used to excuse methods that undermine the integrity of the subject areas we teach.
 
Properly understood, learning outcomes are nothing more than the knowledge or skill levels we expect our students to acquire at various stages of instruction. They are dictated by the body of knowledge being taught rather than the subjective preferences of the teacher. To the extent that an instructor is an expert in the field, she is the person to articulate those outcomes, but the presumption remains that her judgment is compatible with the judgment of other such experts (ceteris paribus). Knowledge is always the result of a peer-review process, and it is through this process that knowledge maintains its status as an objective body of established facts, laws, and principles. Consequently, there should be widespread agreement as to what certain courses should cover and what students should learn.
 
What this means in practice is that teachers are not free to arbitrarily choose the learning outcomes for their courses. Their choices must embody the standing wisdom of the field, measured in terms of expert consensus. This does not mean the outcomes are fixed in stone, since knowledge is subject to adjustment in light of new information; when emendations are required, the experts are the ones to determine what adjustments should be made and where.
 
This being the case, the selection of learning resources and assessments for a course must also be consistent with the specified learning outcomes, and hence with the body of knowledge being taught. Textbooks, articles, and other resources must support the student in the pursuit of the outcomes of the class, and assessments should be designed and calibrated to measure student achievement in substantive ways. Without this alignment of outcomes, resources, and assessments, teaching reduces to guesswork and superstition-the rational process at the heart of teaching and learning breaks down.
 
The foregoing analysis does not, however, eliminate freedom or impose unreasonable uniformity. Learning is a process of discovery, not merely one of justification. While the content of a course depends on the consensus of experts, approaches to communicating that content can and should vary according to the needs of both the students and the instructor. Differently put, while the question of what to teach is constrained by the subject matter, the question of how to teach allows for a great deal of freedom and versatility. This is where academic freedom has always resided, at least where teaching (vis-à-vis research) is concerned.
 
Historically, academic freedom has served primarily as a prophylactic against the interference in academics by the non-expert, the outsider, or special interest groups. It does not and has not sanctioned radical freedom, i.e., freedom to ignore the consensus of the community of scholars, research about what teaching methods are most effective, or the connection between the classroom and the community. Specifying learning outcomes, selecting the appropriate resources to support student learning, and using meaningful assessment measures is no more at odds with academic freedom than the obligation to follow the law is at odds with the principles of democracy.
 
How are learning outcomes identified?
 
Ideally, subject-area learning outcomes are to be identified by the experts in the field, but in accordance with the conventional wisdom among like experts as to what outcomes are appropriate for different levels of instruction. As an historian, for example, an instructor will base her judgment about the learning outcomes for a class on her understanding of the subject area (e.g., what knowledge is fundamental to a more developed understanding of an historical epoch or event), but this is not her judgment alone; as indicated above, it should be consistent with the views of other experts in the field, since a body of knowledge is not the province of any one individual and a consensus of experts is a necessary condition for an area to be considered knowledge at all.[2] Consequently, it is not up to an individual instructor to adjust or delete the identified outcomes for a course once they have been established.[3]
 
General education learning outcomes are by nature trans-disciplinary and require input from experts in multiple fields. Also, their connection to the social and political arenas necessitate more than a strictly academic consideration; it requires an understanding of the demands imposed by society on the individual–the needs for certain skills, dispositions, and values, as well as the common store of accepted truths needed to function in community life (what is termed "common knowledge”). For education to remain relevant to public life, the connections between classroom instruction and the broader social context must be kept clearly in focus.
 
How are GELOs connected to the broader culture?
 
Communication doesn’t take place in a vacuum. The language we use to describe our experience and justify our points of view is part of a fabric of meaning that holds our community together, linguistically and culturally. Our language is public; it is also constantly in flux under the influence of new ideologies, technologies, fads, and fictions. Language changes as ideas change, and ideas change as the result of innovation and history.
 
The language we use in the classroom -- the narratives we construct, the theories we propound – is part of this tapestry that binds us together. Not only do the ideas from academe shape public life, but aspects of the broader culture find their way into the classroom through a reciprocal, semantic exchange. For example, our views of what counts as good thinking are connected to what we count as a problem to be solved, what resources we see as relevant to the solution, and what skills we deem necessary for successful resolution. Each of these components is influenced by elements of the broader culture – the themes of the mainstream media, technology, perceptions of historical events, political challenges to academe, etc. (think about the way discussions of evolutionary theory have been shifted by the political initiatives meant to dislodge it.
 
As education is a form of human communication, the dynamics and principles of communication apply to education. While teaching is never merely a repackaging of the ideas of the broader culture (a reorganizing of popular opinion and belief), our expectations about what students should learn are influenced by what happens outside the classroom. In addition to the obvious point that graduates must have the skills that make them successful in the workplace – and that as the workplace changes, so too do the skills needed to be successful – there is the subtler connection between the classroom and public life: What we teach should help our students come to terms with the complexities and uncertainties of the world in which they live. In other words, teaching should be relevant and meaningful.
 
Not all learning outcomes have a direct connection to this broader context, certainly. Subject-specific outcomes are most closely connected to the theories and themes of course content, which makes the connections to the broader culture less direct. Still, the General Education outcomes must have such a connection, even if subject-specific outcomes are bound to the content of a particular field. As mentioned earlier, GELOs by definition embody the intellectual dispositions and skills needed for success in the world outside the classroom. Information literacy, cultural sensitivity, the ability to identify and solve problems – these things are intrinsically connected to the dynamics of our social, political, and economic experience.
 
This is not to suggest that education is solely or even primarily about preparing students to fill jobs; the richness of the human condition demands more than that. But, whatever the loftier goals of public education may be (preparing graduates for citizenship in the world’s longest democracy being one of them), positioning graduates for financial independence and security is a primary concern since a stable middle class is essential for the long term health of our nation.
 
As educators, our responsibility is to identify learning outcomes in such a way that we balance the dual role of preparing students for the workforce and cultivating their humanity.[4]
 
Is the emphasis on outcomes assessment a threat to instructional quality?
 
No. In fact, just the opposite is the case. By bringing subject-area experts together to discuss the core content of their courses and disciplines, and the necessary materials to support student mastery of outcomes, faculty have an opportunity to reinvigorate teaching, departmental discussions, and professional activities. The identification and assessment of learning outcomes is only one piece of the instructional picture; around the learning outcomes project, instructors can and should build creative and unique approaches to teaching their subjects and engaging their students.
 
Nor is the emphasis on outcomes a threat to traditional liberal arts education or a call for reduction of humanities disciplines to a collection of measurable data. The point here is that the rich, historically important content of our disciplines will rest on a sound foundation of empirical validation only if the learning outcomes essential to our disciplines are identified and evaluated. To think otherwise is to misunderstand what the emphasis on learning outcomes should be about.
 
Is the focus on outcomes related to the Completion Agenda?
 
Yes. Addressing the need to retain and complete more students in the coming years requires that we understand clearly what it is we are trying to teach, what resources are needed to help students get where they need to go, and what instructional strategies promote engagement with our subject areas. Pursuing a learning outcomes assessment project is one way of encouraging a collegial conversation about the core principles of the academic disciplines we teach and building a sound intellectual foundation for innovation and discovery in the classroom. Ultimately, therefore, this project can assist in the effort to identify methods for increasing retention and completion rates.
 
How are learning outcomes assessed?
 
Assessment is a complex process and cannot easily be reduced to one simple mechanism, at least if one is assessing a conceptually rich outcome such as critical thinking. As complex outcomes manifest themselves in a variety of behaviors, multiple forms of assessment are necessary to get an understanding of the extent and depth of a student’s knowledge. For example, that a student can solve calculus problems with great proficiency shows one aspect of her critical thinking ability (see Appendix I), but a host of other assessments is necessary to determine whether her thinking skills are comprehensive enough to count as critical thinking.
 
For relatively simple outcomes such as the knowledge of names, dates, and events, relatively simple assessment methods may be used--and there is little need for multiple methods. To determine a student’s grasp of the basic arguments of Plato’s Republic, for example, a multiple-choice test will suffice (to determine a student’s understanding of who said what to whom, for example).[5] Some subject-area outcomes will count as simple in this sense, but where such outcomes reach higher levels of complexity (e.g., to be able to relate details of one argument in the Republic to others by analogy or by identifying shared premises), the subject-area outcome and the GELOs will overlap. This overlap will necessitate a more complex form of assessment.[6]
 
The advantage of objective (multiple choice, true-false) tests is that they eliminate the problem of inter-rater reliability, i.e., the degree to which different evaluators score an assessment consistently. When qualitative measures are used, such as oral presentations or essays (both of which can give a much clearer picture of a student’s ability to engage in higher-order thinking if used correctly), a great deal of care must be taken to assure that instances of the outcome being assessed are identified and scored in the same way. Rubrics certainly aid in this effort, but getting scorers involved in a detailed conversation about what they are looking for and how they are evaluating responses is also crucial. Identifying an outcome and then leaving individual instructors to assess that outcome without guidance or discussion can lead to incongruities in the data.
 
The lesson here is that academic units responsible for assessing the same outcomes should utilize compatible forms of assessment (or a common assessment) and be clear on how to score the assessment if subjective measures are being used.
 
Is it true that some educational outcomes cannot be assessed?
 
It is true that the long-term effects of education are not easily assessed and sometimes not assessed at all. Outcomes such as ‘inculcate an appreciation for life-long learning,’ for example, clearly fall outside the scope of a particular class, program or degree; assessment would entail a longitudinal approach well beyond the timeline of a two- or four-year degree program.
 
Still, GELOs are good indicators of a student’s developmental progress along certain paths, and these outcomes can be taught and assessed within the framework of formal education. A student who develops an ability to think critically by definition will be more open minded than one who hasn’t (see Appendix I), and open-mindedness is a contributing factor to other, less accessible outcomes such as being a life-long learner. Assessing the GELOs is one, indirect way of assessing developmental outcomes that fall outside the scope of formal education.
 
It is worth adding a cautionary note here. It is sometimes said that some educational outcomes at the course and program level are intangible and hence not assessable – that teaching is a mysterious process that cannot be codified or evaluated in empirical terms. This view is intellectually suspect since it assumes that some kinds of knowledge have no way of manifesting themselves behaviorally. This is what one might call a "metaphysical assumption,” and it is based either on a misconception about teaching or the desire to remain unaccountable for the teaching process.
 
The fact is – or, more cautiously, one fundamental assumption of scientific epistemology is -- any change in a person’s understanding of the world is grounded in the neural circuitry of the brain, circuitry that will influence how a person behaves (directly, or, as is more often the case, indirectly). Some forms of knowledge are more difficult to assess than others, to be sure, (e.g., GELOs versus some subject-specific outcomes), but the idea that knowledge cannot be assessed runs counter to the view that whatever knowledge is, it must at least answer to the description justified, true belief. In other words, the very definition of knowledge entails that it be demonstrable; if a student can’t demonstrate her knowledge of something, she doesn’t have it.[7]
 
* * *
 
Clearly, other questions relating to learning outcomes need to be asked and answered, but the themes considered above should provide a conceptual foundation that allows educators to explore additional questions unencumbered by the typical misunderstanding. In the final analysis, the language of learning outcomes assessment forces a useful re-examination of what we do and how we do it, even if the impetus for adopting this language is imposed from without. By identifying and operationalizing GELOs, mapping them onto course assignments and subject-area outcomes, and aligning resources, educators are in a better position to discover program areas that need improvement, find creative ways of linking course content across the curriculum, and justifying proposals for new and innovative approaches to teaching.
 
In short, the GELO project is a respectable method of revisiting and re-evaluating instructional methodologies and subject areas, and getting this project right can go a long way toward furthering our academic and social interests.
 
Appendix
Examples of Learning Outcomes
 
One of the most common general education learning outcomes is some version of "promote critical thinking,” and for good reason. Critical thinking is synonymous with good thinking, and every educator hopes to promote good thinking through his or her instruction. At the same time, everyone recognizes that good thinking is not bound by a particular subject area; one is a ‘good thinker’ if one can reason well about a variety of different subjects. Thus, the popular conception of critical thinking implies that this kind of thinking is not reducible to a specific subject such as mathematics or chemistry; it is viewed, rightly, as a general ability that applies in a variety of domains. Nevertheless, to say that an educational program or degree should promote critical thinking is terribly vague. Without more detail (or, to use the terminology of science, without operationalizing this concept), educators and the public alike will interpret this idea in different and sometimes incompatible ways. If we are to use ‘critical thinking’ as a GELO, it must be cleaned up and explained. One way to clarify this outcome is to list the more specific abilities associated with good thinking. Here is such a list: Critical thinking implies that one be able to
    1. Classify types of claims as factual/empirical, opinion, descriptive, explanatory, prescriptive, or justificatory (not all of which are mutually exclusive).
    2. Identify the implications of a point of view, argument, or theory.
    3. Monitor the reasoning process; identify and correct errors in reasoning (fallacies).
    4. Distinguish between analogical reasoning, probabilistic reasoning (induction), formal reasoning (deduction), and generalizing,
    5. Extract patterns from data and formulate general laws/rules that account for the data,
    6. Identify a problem, a desired outcome, relevant resources, and a method for solving the problem (when possible)
    7. Understand assumptions and their role in thinking (and communication),
    8.  Analyze language for ambiguity, equivocation, connotation and denotation,
    9.  Adapt to new information contexts.
    10. Imagine alternative scenarios (models) when examining an issue or idea.
Notice that each of these abilities can be assessed; in fact, every critical thinking course must assess these skills and the underlying knowledge they reflect. But they can and should be assessed in other courses as well; a literature course will assess a student’s ability to imagine different scenarios, analyze language, identify implications, etc. Similarly, a chemistry course will promote and assess some but not all of the items on this list within the content discussions specific to the field of study. Each subject will contribute to the critical thinking outcome by getting students to think through the subject being taught; exposure to different forms of thinking across the curriculum is the most fruitful path to a more global form of thinking we designate ‘critical’.
 
Contrast the critical thinking outcome above with one that is subject-specific, such as "State the basic elements of the Socratic Method and demonstrate how that method is applied.” For this outcome, specific to a philosophy class, the list might look like this:
(a) Identify the goals of the Socratic Method,
(b) State the basic assumption(s) about knowledge on which the method is based,
(c) Explain the way Socratic questioning is designed to direct the flow of conversation,
(d) Explain how exposing a contradiction in a person’s point of view can impact the person’s thinking,
(e) Illustrate the method by referring to specific applications in Plato’s dialogues.
The outcome, along with its specific data (a)-(e), is presented in terms of the subject matter being studied in the course. Notice that (a) is connected to vii in the critical thinking outcome, i.e., both require the identification of assumptions, but here we are looking at a specific case – assumptions underlying the Socratic Method. It is easy to see from this connection between the two outcomes how subject-area outcomes support the learning of a general education outcome such as critical thinking. One can also see how assessing a subject-specific outcome can provide evidence of a student’s higher-order thinking abilities. Yet the critical thinking outcome does not reduce to the second, subject-specific outcome; they overlap at certain points, and the GELO is general while the subject-area outcome is specific and context-bound.
 
Given that GELOs have a greater generality than subject-area outcomes, and given that the subscales or data of the GELO cut across discipline boundaries, it is clear that an assessment program that attempts a comprehensive evaluation of a GELO at the course level will fail. Assessing GELOs at the program level is a more reasonable approach, and can be done in a variety of ways (e.g., student e-portfolios or an exit exam). The decision on how to assess outcomes is influenced by how those outcomes are articulated, and consequently should be determined by the subject-area experts at the departmental and program levels (i.e., the faculty).
 

 
[1] This aspect of the relationship between GELOs and subject area content requires much more elaboration than can be provided here. It should be noted, however, that I am not saying that the Gen Ed outcomes can be comprehensively assessed by determining how a student performs on subject-area outcomes within the context of a single lesson or even a single class. The point is that once someone has mastered the GELOs, that mastery is reflected in the performance on specific tasks or exercises, but it is also true that measuring a student’s competency on the GELOs requires evaluating how he performs on novel problems and in novel situations – on tasks he has not encountered previously.

[2] In Florida, the Statewide Course Numbering System (SCNS) categorizes courses based on their descriptions and outcomes in the interest facilitating the process of assigning course credit for transfer students. However, the SCNS does not dictate what courses are developed by faculty or what the outcomes for such courses should be. The SCNS is available at http://scns.fldoe.org.

[3] This may seem a contentious point. The idea, again, is that knowledge is instantiated in a community of experts, that consensus is important, and that deviation from the standard view of course content requires justification beyond wanting to teach something different. If we follow a Kantian principle here and imagined what it would be like to allow every instructor to teach whatever he or she liked, we would immediately see the difficulty. Such a view is conceptually incoherent, since it would undermine the very knowledge we, as experts, claim to profess. Or, we might all agree that allowing experts to teach what they want would lead to a common result, given the nature of their subjects, but this is to tacitly concede the point being made here.

[4] As one of my colleagues aptly puts the point (who wishes to remain anonymous), "A broad liberal arts education ideally adds important intellectual and aesthetic dimensions to life, including fostering a love of learning for its own sake and inculcating an appreciation of the wondrous range of human achievements throughout history. Thus liberally educated people enjoy pursuing intellectual adventures, and they have an enhanced appreciation of the arts. Liberal education makes the experience of life richer, and even though that may not be directly applicable to the workplace, and it may be hard to assess, it’s nonetheless eminently valuable.”
 
[5] E.g., "Which character presents the Allegory of the Cave? Is it (a) Socrates, (b) Glaucon, (c) Thrasymachus, or (d) Polemarchus?”
 
[6] . I am not suggesting that objective (i.e., multiple choice) tests are not up to the task of assessing higher-order thinking skills. Designed in the right way, they can test the ability to draw analogies, identify inductive inference steps, etc. Nevertheless, such tests often have the effect of narrowing the assessment and restricting our understanding of a student’s ability – at least if the objective test is not so intricate that it becomes difficult to understand and complete.
 
[7] . Conversely, if knowledge is not demonstrable, it is also not transferable, i.e., teachable. Any instructor who believes that some kinds of knowledge cannot be demonstrated or assessed should, therefore, leave that subject area out of the curriculum.