Open Options | Evaluation

Evaluating effectiveness
Revision
Other issues

Evaluating effectiveness

Effectiveness should be evaluated.

Technology planning should include evaluation since stakeholders desire and benefit from knowing the effectiveness of implementation. Effectiveness (or ROI) can be evaluated short term or long term. Long-term effectiveness matters more, especially for building capacity. But if the short term is too challenging or unrewarding the long term may be virtually inaccessible. So low threshold solutions may be essential to sustain long term change.

The criteria for evaluation should reflect the needs and goals already identified in the plan.

Evaluation should include broad criteria since schools want to integrate technology across classrooms and teachers. However, at least some criteria should be easily accessible so that stakeholders can enjoy progress and accomplishment in the near future. Just increasing the level of dialog and intention about technology is a worthy activity. The complete realities of total cost of ownership should be evaluated, not just sale prices.

When considering or evaluating open source software the plan should focus on the effectiveness of specific solutions. Some open source software isn't worth the trouble, while other solutions may have a low threshold and/or long-term advantages. Schools should focus on what works and avoid discarding or foreclosing on any effective solution, proprietary or open source.

Do your needs or goals demand a specific, proprietary solution?
Do your stakeholders crave some short-term success in technology integration?
Can you incubate an open source solution long enough to evaluate its actual effectiveness?

[back to top]

Revision

Update and revise regularly.

Schools have limited resources so the plan should include cyclical planning, implementation, evaluation, and revision. It may be unwise to migrate to an open source solution now because of pressing needs or inadequate open source options. But every aspect of every solution should be periodically revisited (even if only to conclude "it works and it's worth the expense").

Figure. Tech planning is re-planning.

When reviewing the plan, stakeholders may re-prioritize their needs and goals. Schools may want to expand, remove, or replace existing solutions. Open source may be the best choice during such changes (e.g. replacing a lab). Open source allows schools to experiment at little cost, and specific programs may mature significantly over time. As schools phase in new technology, older hardware may still be useful by deploying open source. For example, older desktop computers may be recycled as thin clients on a K12LTSP network. (Some current users only buy new Microsoft Windows computers with the kind of network cards that enable this future option.)

Are you willing to re-plan and use data to make changes?
Are you willing and able to consider and experiment with several solutions?
Can you build or shift capacity over time?

[back to top]

Other issues

Technology is a social, legal, & ethical issue, too.

Technology includes far more than hardware and software. Stakeholders may express some social, legal, or ethical concerns related to technology. Such concerns may include: equity and the "digital divide," intellectual property in the digital age, inappropriate content or behavior, virtual monopolies, and balancing technology with other priorities in schools. Technology planning and technology literacy should address these larger issues for all stakeholders, especially students.

Open source offers unique answers to some of these concerns. It may address equity by making infrastructure more affordable or by allowing schools to send free software home with students. The open source community is thoughtful and outspoken in the debates about intellectual property. Free speech and respect are contiguous to the open source philosophy, while open source solutions challenge proprietary models of content filtering and behavior control. (e.g. Open source users can personally control which sites students can access, rather than depending on a proprietary service.)

Schools serve an important role in educating students on these social, ethical, & legal issues.

The implications of these larger issues may be important criteria in the planning and evaluation process. But schools often make decisions individually and without deliberately confronting these issues. Some current users are the exception, and want to engage these issues whenever it doesn't interfere with other, more immediate goals (e.g. supporting effective learning environments).

Are you trying to foster more awareness of the larger issues in technology?

[back to top]

Open Options is a product of the Northwest Regional Educational Laboratory. These materials are in the public domain and may be reproduced without permission. The following acknowledgment is requested on materials which are reproduced: Developed by the Northwest Regional Educational Laboratory, Portland, Oregon.

This Web site was developed and maintained by the Northwest Educational Technology Consortium. The federal funding for the regional technology consortia program ended on September 30, 2005, and no further updates are planned unless additional funding becomes available. However, much of the content is still useful and NWREL will continue to provide access to this site to support educators and to meet its own technical assistance needs.