Assessment Plan
Indicators or outcome measures of goals and learning objectives:Over the past two years the department has begun to assess objectives 1, 3, 4, 5 and 6. Listed below are four examples of ongoing or planned outcome assessments implementing both direct and indirect methods related to those learning objectives. In the next two-year cycle we plan to continue these assessment efforts as well as develop ongoing assessments of learning objectives 2 and 7.
Objectives 3, 5 & 6. Many students in general chemistry experience considerable difficulty with the kind of problem solving skills required to be successful in the course. Small group recitation meetings provide an excellent opportunity for developing these skills, but the traditional activities of going over sample problems or taking a quiz have tended to present more examples of problems without deliberately focusing on the process of solving chemical problems, which is where many students struggle. To address this issue, Professor Richard MacPhail and graduate student Claire Siburt from Chemistry have developed a new model for CHEM 22 recitations in collaboration with Dr. Ahrash Bissell from the Academic Skills Center. The goals of this recitation model are to: develop higher order problem-solving skills that reinforce and expose the course concepts, to engage in collaborative work with other students, and to have an opportunity to present problem solutions orally. The model adapts a “problem manipulation” methodology used by Dr. Bissell in the Academic Skills Center to help students achieve a higher level, deeper view of problem solving (“metacognition”) that focuses on the process and the concepts, rather than just on getting the right answer.
An end-of-course student survey to gauge student perception and overall satisfaction has been carried out two years in a row, with very positive results. In addition, data has been generated that compares the performance on the final examination problems of CHEM 22 cohorts after the implementation of the new recitation model with the performance of cohorts from CHEM 22 and also CHEM 23 prior to the new recitation model. A series of mid-term quizzes were also given during spring 2007 to provide a longitudinal assessment of learning gains over the period of the course. The results of these assessments are being analyzed to judge the effectiveness of the recitation model in moving the students toward the stated goals, especially the primary goal of enhancing problem solving skills (no direct assessment has been made of the team work and oral presentation aspects of recitation). If a significant increase in goal attainment is seen in the cohort comparison and the learning gains assessment, then this will provide an impetus for improvement and possible extension to other chemistry courses beyond the very favorable student perception of the recitation model. Some discussion and graphs related to the survey results can be found here.
Objective 3. Written communication skills are first stressed in a comprehensive way in the physical chemistry laboratories CHEM 167L and CHEM 163L, and then again for a subset of BS majors in CHEM 168L and CHEM 180L. Oral presentation skills are incorporated in CHEM 168L and CHEM 180L. Grading rubrics have been developed for both written and oral presentations within these courses, and contribute significantly to the overall course grade. For example, in CHEM 163L in spring 2008, 40% of the possible points for all assignments in the course came from scores on written reports and reviews, and the 23 students (of 48 total) who attained an overall grade of A or A- generally scored 80% or better on the writing portion.
This past year we have added a new half-course CHEM 198 for the subset of students pursuing Graduation with Distinction. Students in CHEM 198 receive additional training in communication skills, and they give an oral presentation and submit a research thesis that are evaluated to determine whether they receive Graduate with Distinction status. CHEM 198 thus provides an opportunity for a direct assessment of communication skills prior to graduation, with the outcomes being the capstone thesis and presentation. Our plan is to develop a grading rubric for oral and written communication skills in CHEM 198 (based in part on the successful BioTAP thesis assessment protocol developed in the Department of Biology) that will allow for a consistent comparison of learning gains across the different curricular tracks involving CHEM 163L, 167L, 168L and 180L. This will require a collaborative effort between the instructors of these courses both to develop the appropriate rubrics and to set expected levels of performance. Additionally, the instructors of CHEM 198 hope to involve Duke alumni as readers for Graduation with Distinction papers, which would provide for an external assessment of writing skills, in particular the ability of the student to communicate their work to a scientifically literate, but non-expert audience. The results of these various assessments in CHEM 198 can then be used to tune our training in communication in the earlier courses to improve the outcomes.
Objectives 1 & 5. To many students, general chemistry can appear as a collection of disconnected concepts and facts, an appearance that is enhanced by the way the topics are divided up in the chapters of typical general chemistry texts. Similarly, many students see general chemistry and organic chemistry as separate entities and fail to see how the underlying concepts and problem solving skills introduced in general chemistry carry over to organic. Over the past two years our one semester introductory general chemistry course CHEM 23 has been restructured by Professor Eric Toone as a first step toward a more comprehensive restructuring of our introductory chemistry courses aimed at dealing with these issues. Goals of the new prototype CHEM 23 are to enhance problem solving skills and preparation for organic chemistry, to stress the interrelationships between different topics in general chemistry and their relationship to organic chemistry, and to communicate the relevance and excitement of chemistry, all by placing the chemistry concepts in the context of an overarching problem, in this case chemistry of the cell and its relation to cancer.
We plan to assess the effectiveness of this new course in several ways. Student surveys at the end of CHEM 23 and at the end of the first organic chemistry course CHEM 151 will be used as an indirect assessment of the students’ perception as to whether CHEM 23 enhances the preparation for organic chemistry, and whether students have developed a better appreciation for how chemistry can address large, complex problems like cancer. Direct assessments of problem solving skills will include comparisons of grades in organic chemistry between groups of students coming out of different introductory chemistry courses, e.g. CHEM 23 versus CHEM 22, and through specific embedded exam problems given in both CHEM 23 and CHEM 151. The results of these assessments will be used to further tune the new format of CHEM 23 and to inform our other plans for curricular restructuring.
Ancillary Assessment and Impact:
An off-cycle introductory general chemistry CHEM 21/22 sequence was introduced in spring 2007 in response to a study by the Dean of Trinity College. The findings of the study indicated that students taking CHEM 21 and MATH 25 simultaneously during their first semester were at significant risk of performing poorly in both. The administration has been assessing whether delaying the start of introductory chemistry for this group of students enhances their performance and an interim report is available here.