Week 3: Chapters 10-13

Section 3: Evaluating, Implementing and Managing Instructional Programs and Projects Due: 11/21

All too often instruction is developed with little thought as to how evaluation of learning will occur, or the evaluation of the effectiveness of the instruction. Evaluation is usually an afterthought - an add on. It's time to start thinking about evaluation on the front end of the instructional design process.

Additionally, there is resistance to the adoption of technological innovations and we must develop an understanding of why and how to facilitate the adoption of innovations with an understanding of why there is resistance.

For this week's reflection activities, I would like for you to concentrate on the following:

1. Chapter 10 discusses evaluation in instructional design and provides you with two evaluation models, the CIPP and Kirkpatrick models for evaluation. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction.

I first researched the Smith and Ragan model for summative assessment. I have listed the steps of this model, followed by how I would implement it to evaluate my instruction.

· Determine goals of evaluation – What do I want to find out as a result of this evaluation—for example, “Did my students master how to determine meter in poetry?”

· Select indicators of success – What specific details will be used to determine whether or not the instruction was successful

· Select orientation of evaluation – Quantitative or qualitative data

· Select design of evaluation – Determine what data will be collected, when the data will be collected, and the conditions under which data should be collected in order to answer the evaluation questions

· Design or select evaluation measures – Include several different categories to measure the success of the instruction.

· Collect data – Conduct the actual evaluation

· Analyze data - Descriptive statistics (e.g. means, range, frequency) or inferential statistics (differences between two instructional programs or from pretest to posttest within the same program)

· Report results

The second model I researched was the Flagg model for formative assessment. This model includes the following steps:

· Needs assessment – Determine the reason for the instructional program;

· Pre-production formative evaluation – Utilize target audience and teachers to make design decisions about content, objectives, and production formats

· Production formative evaluation – revise the instructional program based on target group feedback

· Implementation formative evaluation – Determine how well the instructional program operates with target learners in the environment for which it was designed via field-testing

2. Think about a technological innovation within your social system that was recently introduced (social system = place of work, home, etc.). Any innovation has what is known as perceived attributes - relative advantage, compatibility, complexity, trialability, and observability. Describe these attributes of the innovation and which of the attributes assisted and/or hindered adoption of the innovation. For example, was it not compatible with the existing beliefs of the system or was it overly complex to use or understand?

The technological innovation that came to mind for me was my school’s adoption of the C-Scope curriculum program. It was actually implemented last year, but core subject teachers had the option of using it or not. This year, however, all core subject teachers are required to use it.

Perceived attributes of the C-Scope system are as follows:

· Relative advantage – The main benefit of switching to the C-Scope system was that it would provide a standard curriculum for all teachers to follow. It would also clearly identify the specificity to which teachers should teach concepts.

· Compatibility – The C-Scope system was presented to teachers as a tool that would help align curriculum from 9^th through 12^th grade. It was also presented as a way to ensure that all TEKS were being covered in order to prepare students for the TAKS tests. Since the high school had previously received an “Unacceptable” rating (due to math and science scores), this was a major point in favor of adopting the system.

· Complexity – The C-Scope system was presented as being very simple to use and extremely user friendly. In addition to example lessons, the system also contained handouts and assessments that could be utilized to evaluate mastery of the material being presented.

· Trialability – The 2009-2010 school year was basically the trial period for the C-Scope system. Math and science teachers were required to use it, while it was optional for English and social studies teachers.

· Observability – The school’s 2010 TAKS scores were improved in the areas of math and science, resulting in an “Academically Acceptable” rating for the school. District and campus administrators credited the increase in scores to the successful implementation of the C-Scope system and made the decision to require all core subject teachers to use the system for the 2010-2011 school year.

Many of the teachers at my school are very unhappy with the C-Scope system. In my department, we have noted that the assessments are very poorly written. Many questions are somewhat ambiguous or have more than one option which could be correct. Also, we have noted that the assessments often include questions that do not relate to the TEKS specified for the unit. The biggest problem that my department colleagues and I have with the C-Scope system is that the English portion is not yet complete!! As you can imagine, this makes it very difficult to successfully implement the system. Other complaints include the fact that topics are only covered superficially (even though they will be on the TAKS test), and the pacing calendar leaves almost no time for creativity or for re-teaching if needed.

3. Chapter's 12 & 13 focus on project management and how to manage projects when resources are scarce. You have been assigned to develop a series of professional development sessions focusing on technology use in the classroom for teachers. How will you use Situational Leadership to facilitate this project?

To use Situational Leadership for this project, I would:

• Define and describe the group tasks.
• Determine the competence and commitment levels for each group member. This will ensure that all group members have the skills and the motivation they need to complete their assigned tasks.
• Choose a leadership style that is appropriate for the development levels of my group members. Determine if the group members need more direction and oversight or can be allowed more autonomy and freedom.
• Communicate with all group members to ensure that everyone is on the same page with regard to the tasks and the overall project.
• Stay in contact with group members as they complete each assigned task.
• Evaluate the effectiveness of my leadership style to ensure consistency and effectiveness.