EETT Evaluation Lessons Learned from NW States' Projects 2003-2004
Since the spring of 2003 NETC has been providing technical assistance in evaluation design to EETT grantees in Idaho, Montana, and Alaska. The assistance has been delivered via one and two-day workshops to 52 districts in Boise, Helena, and Anchorage. The content of the workshops focused on developing logic models and causal maps, leading to improved project evaluation plans.
Because of strong positive response received during the workshops, NETC wanted to know more about the longer term results for districts who received this technical assistance. Were grantees able to apply the content to their projects? If so, how and what difference did it make? How did it impact their projects?
To help answer these questions in spring and early summer of 2004, NETC talked to project directors and evaluators, SEA leadership, and read annual reports submitted by grantees. In addition, during a half day "Lessons Learned" workshop at NECC in New Orleans we gathered comments from the 30+ attendees.
Following are some of the lessons gathered from this process.
- Learning to apply logic modeling and causal mapping to EETT grant projects provides multiple benefits:
- More focused projects which helped target time and money to achieving key outcomes
- Better tracking and understanding of project effects
- Increased ability to explain projects and communicate results to stakeholders;
- Developing this shared understanding within project participants leads to better fidelity of implementation; fewer "pockets of excellence"
- More consistent data collections and reporting of results to SEA
- More efficient collaboration between project evaluators and project leadership
- Promotion of state-wide evaluation of grant expenditures, which in turn supports federal understanding of EETT program effectiveness
- Improved chances of sustainability of projects beyond grant period
- Districts have difficulty reporting results
- There is a need for ongoing support to grantees in evaluating projects
- One year grant is too short for grantees to show achievement gains