Center for Teaching Excellence Newsletter
Frostburg State University
April 2008

CTEAG membership: John O'Rorke (POSC), Jodi Welsch (EDUC), Tom Hawk (MGMT), Tracy Edwards (GEOG), and Jim Limbaugh (Provost Office).

Something to Think About: "To doubt everything or to believe everything are two equally convenient solutions; both dispense with the necessity of reflection." ~ Henri Poincare

 

Using Assessment to Refine Programs
by Doris Santamaria-Makang

In higher education institutions there are different ways to determine whether assessment is facilitating the level of decision-making in the process for improving programs. One way is to see if there is evidence of decisions based on assessment results that influence the improvement of programs. Another way is to examine if the assessment process leads programs towards rigorous self-evaluation. Still, another way is to assess if the institutional culture reflects a changing tradition framed by evidence-based decision-making practices. When speaking about program refinement, it may be important to discuss why any of these approaches would be valuable, and what it looks like when assessment results are used for meaningful decision-making to alter programs' practices.

In the College of Education at Frostburg State University, we have instituted an assessment system that specifically addresses the use of assessment results to facilitate the fine-tuning of programs. It is a comprehensive system with a cyclical structure designed to document candidates' successful completion of program outcomes by gathering evidence, interpreting results, and implementing change. As such, this structure conveys a systematic process for evaluating courses, programs, clinical experiences, and candidate performance. Each program has an advisory council that meets at least once a semester to review course, program, and clinical experience data and make recommendations for modifications and/or adjustments.

The true value of this assessment system is that it provides faculty with a format for quality review of the effectiveness of the programs in terms of their impact on student learning. Although we are constantly redefining our assessment system's practices, we have learned numerous lessons in the process. We learned that by critically assessing assessments in terms of their effectiveness for reaching programs' outcomes, it allows faculty to implement assessments in a manner that gives programs the flexibility and autonomy that faculty need in order to make their programs successful. It also allows for the implementation of a culture of self-study in which the process and decisions can be sustained beyond any accreditation visit.

An internal characteristic of our assessment system is that it promotes an innate curiosity among faculty about what works or does not work in a program. As such, we learned that the assessment system needs to be institutionalized to allow for a process with a shared purpose among faculty to collectively commit to improving the quality of the programs. This commitment can be seen in decisions made to improve student learning as well as the assessment process. While this short article does not focus on how the characteristics of this assessment system are implemented, the following example illustrates briefly how a segment of this process was carried out in the Early Childhood Program:
Through a regular review and analysis process of Early Childhood Praxis II scores, the Early Childhood Coordinator and faculty noted that the passing rate for candidates in Early Childhood content for the 2002-2003 periods was 80%. This passing rate was relatively low compared to the passing rate for Elementary program candidates. A review of the content knowledge required for Praxis II led to changes in the Early Childhood curriculum. The pass rate for 2005-2006 is 97% and the Early Childhood program was nationally recognized by its Specialized Professional Organization (SPA) review with no conditions.

By nature our assessment system is a dynamic process, and although it has come a great distance and in some ways has been a model for the University's assessment efforts, we realized that it constantly needs to be refined. We learned that the assessment system has to be subject to periodic revisions dictated by new findings emerging through the data collection process, and by recommendations from current and relevant research about the use of assessment for program improvement.

The most important lesson we learned is that the 'refinement of programs that leads to student learning' is not an item that we can buy directly from a manufacturer's store. Instead, it is a complex practice that requires time, expertise, and commitment to a system that allows for quality review of the program's assessment process; that is, a systematic approach in which the data provides opportunities for reflection upon student and program outcomes.

Note: A graphic representation of the College of Education Assessment Cycle is described in the "Assessment Plan" page of the College of Education website at: http://www.frostburg.edu/colleges/coe/

What Do You Think? In our last newsletter we asked you to respond to the following: "What is your greatest challenge as an instructor? " Here's what your colleagues said:

" "Trying to provide motivation to the millennium generation students. It seems that they have been spoon fed so much, that they don't even know how to try to achieve on their own; and I believe a major reason is the lack of motivation. Why bother to try when someone will "fix it" for me anyway? It is an interesting & challenging dilemma."

" "There is little agreement on and coordination of the content development flow within the program. Even when there is the appropriate content development flow, the pedagogical approaches used by those in the prerequisite courses is ineffective in developing the understanding and practical use of the content and skills."

" "Working in an environment in which not all my colleagues challenge their students creates a set of expectations on the part of students that I do not and will not honor. Hard work earns the grades - how do I impress that upon them?"

" "I teach both graduate and undergraduate counseling students. Counseling is an art. One of the things that I found out about myself as a photography student is that I do not have an eye. I think that some people are not counselors. I think that there are some things that can not be taught. You have it or you do not. There must be some raw intangible essence there. Some might call it talent. Whatever it is, some of the students do not have it and are not going to have it. Trying to break the news to a student who has his or her heart set on being a counselor that at least at this particular time, the student does not have the skill set, despite my and his or her best efforts, is my greatest challenge."

" "There is little or not faculty discussion and agreement on the appropriate and high level of expectations and challenge in the courses across programs."

" "The thing I've struggled with since I've begun teaching is my relationships with the students. Effective instructors seem to be able to develop meaningful connections with students and these relationships have positive effects on the student's performance and the instructor's teaching. I have high expectations for my students and strict requirements for my courses. It seems that these characteristics often intimidate students and create a barrier between us. I've been working over the past few semesters to find ways to connect positively with students, to convey my concern for their success and to let them know that I really am a human being! "

Look for more questions to ponder in the fall!