Project Evaluations |
|
| Evaluation 2003-2008: Science Knowledge and Science Attitudes Progess Reports: 2006 | 2005 | 2004 |
|
| Evaluation 2000-2003: Curriculum Dissemination Methods Progress Reports: 2002 | 2001 |
|
| Evaluation 1997-2000: Student Drawings and Sentence Completions Progress Report: 1998 |
|
Rubrics |
|
| Rubrics Overview | |
Evaluation 2003-2008:
Science Knowledge and Science Attitudes
Science Knowledge: Pre/post science knowledge tests were collected in 2006-2008 to test the effectiveness of the curricular materials.
| Lesson Title | Pre Test | Post Test |
|---|---|---|
| Levers in the Body go to Levers Unit |
47% |
83% |
| N=95; p<.001; r2=.53 | ||
| Sedentary Stan go to Diabetes Unit |
61% |
78% |
| N=56; p<.001; r2=.22 | ||
Note: middle school students
The following are more examples of pre/post test results.
Science Attitudes
At the beginning of the 2004-2005 school year we collected data from over 3,000 middle school students regarding attitudes toward science using six published scales. We are empirically re-evaluating the psychometric properties of these instruments including the Draw a Scientist Test (DAST), the Images of Science and Scientists Scale (ISSS), the Test of Science Related Attitudes (TOSRA), the Women in Science Scale (WiSS), the Scientific Attitude Inventory (SAI), and the Simpson Troost Attitude Questionnaire (STAQ).
Our preliminary analyses showed that factor structures did not match original authors’ claims. Using exploratory and confirmatory factor analyses, two- and three-factor solutions were found for the WiSS, ISSS, and SAI instruments. Several items from these instruments that did not fit the factor structures were removed. These shortened instruments with improved psychometric properties should prove to be more feasible for researchers and administrators. Additionally, our comprehensive review found that the majority of instruments have not undergone rigorous psychometric measures but are still being used for program evaluation. Further, only a minimal theoretical basis or historical perspective is provided by researchers in defining “attitude” (particularly “science attitude”), and why or how attitudes should be measured. Read more under our Peer Reviewed Publications link.
Evaluation 2000-2003:
Curriculum Dissemination Methods
A controlled trial evaluation was conducted with four schools to test the best dissemination method. Data showed that in-school support by study staff working with school teachers was necessary to disseminate and increase utilization of the Positively Aging® teaching materials. Having the materials available on a website alone and providing summer training was insufficient. Further, utilization of the teaching materials remained low, in spite of the in-school efforts. Several barriers to successful implementation were identified including student turnover, teacher turnover, competing programs, state and district mandates, and standardized testing.
Read more under our Peer Reviewed Publications link.
Evaluation 1997-2000:
Student Drawings & Sentence Completions
A controlled trial evaluation was conducted with two schools. Qualitative methods included using children's drawings to capture changes in views on aging. The implementation school data showed an increase in positive views and a decrease in negative views while control school data remained constant. Data findings were statistically significant and demonstrated that the Positively Aging® materials can move children to a more positive view of aging. Read more under our Peer Reviewed Publications link.
You may download our coding sheet and instructions:
| "Help the N.I.A." Drawing Activity (Read more about this activity.) | ||
| Student Handout | Coding Sheet | Coding Instructions |
| Sentence Completion Activity (Read more about this activity.) | ||
| Teacher Handout | Coding Sheet | Coding Instructions |
Teacher & Student Opinions
One way to qualitatively evaluate the curriculum and program is to ask the opinions of teachers and students.
Example Quote: "With this program, any teacher can pick up a lesson and teach a health topic." - High School Special Education Teacher
Read more quotes from teachers and students here.
Student Grade Cards
Another alternate for curriculum evaluation is for the students themselves to evaluate the curriculum! Explain to your students that they always receive grades for their work, now is their chance to grade the lesson. They can give the lesson/activity an A, B, C, D or F and explain why it deserves this grade. Teachers can collect the grade cards and analyze the comments for patterns or common themes.
You may download a sample student grade card here.
To assist teachers with evaluating their students, rubrics are provided as examples of how to assess the quality of the work and processes by which the children are learning. The percentage grade applied to each rubric may vary with the skills being tested, the format for the activity (oral vs. written), and the time of year the questions and activities are used. The evaluation rubrics may be weighted to obtain a percentage grade by applying different scales - examples of scaled scores are also provided in this section.
Introduction: Evaluating the effectiveness of the curricular materials and changes in the students' knowledge, attitudes, and beliefs is central to accomplishing the objectives of the TEI lessons. In this lesson, we provide some examples of evaluation rubrics you may use in your classroom to assess your students. A rubric is a guideline with specified criteria for assessing the abilities and knowledge of your students. The rubrics are shared with the students before the activities so that they will understand how they are being tested and evaluated. The rubrics provided here are meant as suggestions. Teachers may adapt their own evaluation rubrics for assessing their students when using the TEI instructional material.
To the Students: "The following are copies of some of the evaluation rubrics we will use with the TEI teaching material this year. A rubric is a system that allows you to know what is expected of you and how you are learning, improving, and progressing in class. Before each learning activity, you will be told which scaled rubric (score) we will use and what skills we will be evaluating. As your teacher, I believe you will appreciate this method of grading your skills. So often, you do many of these things well but are not given credit for them. Enjoy!"
Reflection: "Without help, write down the (concepts, exercises, or observations) we made together in class yesterday."
0 = None written 1 = Incorrect Information 2 = 1 correct item 3 = 2 or more correctly recalled items
Participation - Group Checklist : Does the Student:
• Ask questions? • Make appropriate comments? • Take notes? • Complete tasks?
Participation - General:
0 = Does not participate 1 =
Participates inappropriately (specific rules for participation set prior to each activity) 2 = Participates appropriately for assigned task
Contributions to the Group Checklist : Does the student:
• Ask questions for the group? • Keep the group on task? • Encourage others?
Group Skills and Participation:
0 = Interferes with group 1 = Does not participate 2 = Participates some of the time 3 = Always participates
Group Skills and Cooperation:
0 = Does not contribute to the group 1 = Sometimes contributes 2 = Consistently contributes to the group
The most common process skills require that students be able to use a broad range of content knowledge to:
1. Identify problems 2. Formulate hypotheses 3. Design experiments 4. Make observations 5. Collect data 6. Organize data 7. Interpret data 8. Formulate conclusions 9. Classify and generalize 10. Make predictions 11. Report results
Examples for rubrics for some of these process skills include:
Understanding the Problem:
0 = Student fails to restate problem in own words 1 = Student restates problem but misinterprets 2 =
Student restates problem and demonstrates correct understanding
Designing the Experiment:
0 = No experimental design 1 = No comparison of variables 2 = Comparison made but lacking data collection 3 = Comparison made with sufficient data collection
Making Observations:
0 = No observations made 1 = Only 1 or 2 observations made 2 = 3 or more observations made 3 = 3 or more observations made with supporting data collection
Data Collecting and Reporting:
0 = Student fails to collect any data 1 = Student describes observations in a disjointed manner 2 = Student produces data tables without meaningful labels 3 =
Student produces data tables with meaningful labels, but inaccurate recording of data 4 =
Student produces data tables with meaningful labels and accurate recording of data
Drawing Conclusions:
0 = No conclusions made 1 = Conclusions made but not supported by data 2 =
Conclusions supported by data by student fails to present the evidence 3 =
Conclusions supported by the data and student provides evidence
Making Predictions:
0 = Student makes no predictions 1 = Student makes one prediction 2 = Student makes one prediction per observation 3 =
Student makes one prediction per observation and the prediction is accurate/correct
0 = No Response from student. 1 =
Student response does not answer question or parts to a question are not answered. 2 =
Student answers questions without supportive detail; student may or may not respond in complete sentences. 3 =
Student processed questions with some supportive detail; student may or may not respond in complete sentences. 4 =
Student processed questions thoughtfully with supportive detail but did not answer in complete sentences. 5 =
Student processed questions thoughtfully with supportive detail or justification and the responses were written in complete sentences.
In addition to the rubrics noted above, scaled scores may be provided to weight the numerical values. These weights may then be used to provide traditional averages based on a 0-100 rating scale. Students should be aware of the scaling values assigned to the rubric numbers. Examples of different scaling schemes are provided.
Scaled Scores Rubrics
Teachers are encouraged to develop and utilize their own scaling as their classroom needs evolve.
