Assessment Matters

In Assessment Update (Sept./Oct. 2009), Elizabeth Greville outlines the specific reasons why Assessment professionals discourage course grades as effective measures for assessment, i.e.  achievement of program-level student learning outcomes, however she suggests that:

With careful, deliberate, and disciplined organization of teaching, evaluation, and grading, outcomes assessment does not have to be a separate activity from grading.

The approach she describes necessitates that faculty make a significant front-end commitment in the articulation of program outcomes as well as identifying course objectives and assignments that can effectively inform these outcomes. If a program can take advantage of specific student products that are embedded within courses, programs can capitalize on students’ motivation for grades/credits and the natural reliance on faculty to make expert judgments about student achievement. As she states,

What matters is clarity about what is being assessed and the basis on which a judgment will be made. If a grading system is organized with explicit clarity about how those judgments are made, grading and assessment can coexist.

You may also find the Noel Smith article, “Challenges to the Credible Assessment of Learning” (specifically Challenge 3: Separating Instruction and Assessment, pg 7) in this issue worth your time.

We hope you find something you can take away from these articles.
Do not hesitate to contact the Assessment Office if you would like to discuss them further.

Two opposing postings we should consider!
Laurie Fendrich’s ‘You Will Be Held Accountable‘ continues her lambasting of outcomes assessment in higher education. She criticizes how “outcomes assessment actually plays out in practice is appalling” and is “detrimental to higher education itself.” In her context, which appears to be more related to across the board ‘testing’, we tend to agree. At UNR, the Assessment Office promotes a faculty-driven model that asks individual programs to use data they collect within their program, encourage faculty to reflect on its meaning and use their judgment and expertise to direct program modifications.

The second posting, Kevin Carey’s $$$, Assessment, Etc., considers how the dearth of information about learning in higher education puts institutions at risk in the current economically stressed times. Click on his new column to read his expansive commentary. He makes concessions to Fendrich’s major points (unavoidable costs, limitations, and inaccuracies), but asks us to be practical – TOO! “. . . perfect and total information can’t be the enemy of good.” It’s difficult to argue that programs should not strive to improve, but we suggest programs need to make changes on the basis of information, not in its absence.

Enjoy these commentaries and join the Chronicle’s Brainstorm weblog as well as our own.

During the 33^rd Annual Conference (Nov. 12-14, 2008, in Pasadena,) of the California Association for Institutional Research, “Making Good Decisions in: Challenging Times”, educational effectiveness as related to accountability and accreditation assumed center stage.

The Keynote by Teri Cannon, Exec. Assoc. Director for WASC, stated, “. . . the pressure for greater accountability and transparency will not go away with a new administration.” She when on to say:

• The movement is bolstered by the frequent release of new data that show that
  many college graduates cannot read and write at what was once considered the
  college level and do not know the basics of math or geography;
• It is pushed along by an increasingly vocal employer lobby and business
  community that wants work-ready graduates who can communicate effectively
  orally and in writing, can work well as members of a team, and have
  competencies for the diverse and increasingly globalized workplace; and
• It is also driven in large part by evidence that the US is falling behind in
  international competitiveness as a result of our declining educational
  effectiveness as measured by several recognized indicators of educational
  achievement. (Full Address)

She briefly mentions the Voluntary System of Accountability, Learning Accountability from Bologna (also Adelman’s The Bologna Club), and the 2008 Higher Education Opportunity Act in laying out challenges for the future of WASC accreditation.

Review the Conference Program including links to other keynotes, panels & presentations.

During a meeting of officials from liberal arts colleges, associations and foundations (e.g. AAC&U and CHEA), Teagle consultant, David Paris, outlined the meeting’s goal as, “being able within the next 3-5 years to say confidently to the public and public officials we have engaged in systematic and even systemic improvement [by] harvesting … some of the 1,000 flowers” now being nurtured on individual campuses.

If enough colleges get serious about assessing student learning on their campuses, being among the “1,000 flowers” that higher education leaders say are blooming, a fruitful “competition of ideas” will emerge, argued Robert J. Thompson Jr., a professor of psychology at Duke University who is leading a joint Teagle/Spencer initiative designed to help get major research universities on the assessment train, which they have been slow to board.

A notable caution was also raised:

“My concern is that our focus on improving student learning needs to be driven by institutional mission rather than an effort to appease external audiences who may not understand our missions,” said Peter H. Quimby, deputy dean of the college at Princeton University, who expressed reservations about student learning assessment broached in the context of accountability. “When we buy into the market-oriented rhetoric of accountability, value added, and cross-institutional comparisons in order to placate others, we run the risk of making it harder to engage faculty members in conversations that are both meaningful to them and helpful to our students.”

Read the full Inside Higher Ed article (Spreading the Gospel on Student Learning) as well as add your comments to the ongoing conversation.

The University of Nebraska announced their superior achievement in value-added learning based on students’ performance on the Collegiate Learning Assessment (CLA). The CLA is one of three performance-based measures (tests) of student learning approved for use on the College Portrait web template (Voluntary System of Accountability-VSA) to which UNR has signed on. The other tests are the Measure of Academic Proficiency and Progress (MAPP from ETS) and Collegiate Assessment of Academic Proficiency (CAAP from ACT). Participation in the VSA will require UNR to administer of one of these assessments (as well as a survey such as the National Survey of Student Engagement – NSSE) to freshmen and seniors over the next 4 years.

The use of a single test to assess general education achievement is controversial. Although standardized tests may seem efficient in their simplicity, there is a downside. Not only are there limitations (Trudy Banta), they bestow an illusion of precision to the assessment process that makes them susceptible to misuse, and not only by institutions, as this Inside Higher Education article Let the Assessment PR Wars Begin illustrates.

Our old friend, Cliff Adelman, adds a his choice comments:

If U.S. higher education really believed that the CLA (or any other test) was a valuable measure of what it expects every student to learn, was, in fact, the ultimate measure of what it does, then it should require all students—and not just unrepresentative samples—to take and pass the exam in question as a condition of graduation. Each school can set its own cut score (there are standard psychometric techniques for doing so), and give students 3 shots to pass.

At that point, the credibility of the enterprise will rise. Until then, what we are doing is just a hollow show. Besides, our students don’t get degrees in “critical thinking” or “making and breaking arguments.” They get degrees in engineering, nursing, anthropology, etc.—and that’s the way we organize our faculties and curriculum as well.

Read Cliff’s full commentary, the wide ranging opinions many others and get involved in the online discussion yourself by adding your views to the conversation.

More food for thought!
Check out the oped by Charles Murray (Yes of Bell Curve fame), For Most People, College Is a Waste of Time from the August 13, 2008 Wall Street Journal.

The Higher Education Act may finally be re-authorized following a 40-4 vote of the Conference Committee last night, July 29, 2008. One provision in the legislation, which influences how accrediting agencies (Northwest Commission on Colleges and Universities accredits UNR) go about reviewing campus-wide assessment of student learning, will:

“Bar the U.S. Education Department from issuing regulations (governing higher education accreditation) designed to ensure that colleges are measuring student learning outcomes, . . .”

Other provisions will mandate transparency in areas, such as tuition and financial aid, but only time will determine how the National Association of State Universities and Land Grant Colleges (NASULGC) & American Association of State Colleges and Universities (AASCU) initiative, the Voluntary System of Accountability’s College Portrait will evolve.

The major provisions are highlighted in today’s Doug Lederman’s Inside Higher Ed article,
7 Years, 1,158 Pages. . . and Almost Done.

A Chronicle of Higher Education article, House and Senate Negotiators Approve Compromise Higher Education Act, details increased federal oversight.

During a 2008 Council for Higher Education Accreditation’s (CHEA) Summer Workshop Panel discussion, the CHEA president [Judith Eaton], an Education Department official, a key Congressional aide, and two accrediting agency leaders debated Eaton’s vision and, more fundamentally, the current state of the tension-filled system by which the federal government, accrediting agencies, states and colleges and universities seek to ensure the quality of education provided to students.

Panelist discussed the divide between those who believe federal rules should govern accrediting agencies standards for student learning and those who believe, “Institutions should be able to define for themselves what participation in postsecondary education provides to an individual.”

Senator Lamar Alexander’s Congressional Aide, David Cleary, warned that just because Congress has stepped in to limit DOE regulation . . . does not mean college leaders should assume accountability is not forthcoming. Cleary said, “I think we’re talking about a five-year time frame.”

Read the full description of the Inside Higher Ed article.

DOE may still issue regulations for accrediting agencies’ oversight of colleges’ student learning outcomes. Secretary of Education, Margaret Spellings, under pressure from Congress had agreed not to issue such regulations until the re-authorization of the Higher Education Act. As the final vote on this Act is expected by the July 4th recess, concerns mount among higher education officials, lobbyists & Congressional aides that the Department will take a ‘Last Bite at the Apple.”

With the clock ticking on the Bush administration’s time in office, many higher education officials have wondered whether (and in some cases feared that) Spellings and her aides might make a final push to institute policies that might instigate change in higher education.

Read the full article in today’s Inside Higher Ed.

Read/Respond – Cliff Adelman Comments, “You can’t sit on your hands on this one: . . .The present prospects are utter folly.”

In today’s Inside Higher Ed, Doug Lederman details the status of a House & Senate Conference Committee’s work to reconcile the HE Act by Memorial Day. Work not only remains on major difference between the House and Senate version, but one Congressional aide says, in reference to an expanding number of lists intended to improve consumer information & institutional transparency, “It seems like it is going to serve to either confuse the hell out of people or give people a false confidence or a false sense of danger.”

Regarding DOE’s growing involvement in accreditation matters, “One other closely watched area in which the draft legislation offers a compromise is in the area of accreditation, where the circulated discussion draft would continue to bar the U.S. Education Department from promulgating regulations governing student achievement (and additional areas such as curriculum and admissions practices) but would allow the agency to regulate in realms such as faculty, facilities, and student support services.”

Read the full Article & don’t miss Adelman’s commentary.

Doug Lederman’s Inside Higher ED article (April 15th), “Margaret Spelling, Where Are You?“, describes the extensive as well as varied approaches being undertaken by faculty and institutions accredited by the North Central Association to address external demands for accountability.

“We try to not just be generating data because someone at the state level is asking for it . . . We try to process it in a way that is going to serve the institution…. And many of the things we have undergone internally to improve our own processes and student learning have helped us to be able to respond quickly and agilely to state mandates.” – Stephanie Booth, Kent State Assoc. Provost

Read the full article and Cliff Adelman’s interesting comment!