Evaluating Technology Tools for Your Classroom

By Judi Mathis Johnson

With more than 15,000 titles of educational software programs, more than 200,000 companies designing software for handheld computers, and millions of Web sites, educators face the daunting tasks of choosing the best product for the job and then using it effectively. Healthy doses of skepticism, appropriately applied, will lead your technology evaluation process to good software applications and easy-to-use equipment for the classroom.

The evaluation process can be simplified by asking vendors the right questions and taking lessons from the researchers and reviewers whose job it is to sort out the merits of varying products. Education is one of the few disciplines that has no mechanism for identifying the "worst-in-class," so consumers must be alert. Read product descriptions carefully and ask critical questions like the following:

What is new and wonderful about this product?

New software announcements, whether in print or Web-based publications, usually contain a good deal of marketing hype. Beware of grandiose claims about the effectiveness of new products that you know have not been around long enough to earn a successful track record. Too often over-hyped software products prove to be "vaporware." If evidence is presented that a new title has proven successful in classroom implementations, the odds are it is a beta test version that is being described, not a finished version ready to be sold. Make sure the product you are considering is a final version--one that truly exists.

Many vendors of educational products offer listservs to keep you informed about upcoming products. If you decide to sign on, provide an e-mail address that is different from your school or work account. Be careful when inquiring about new products and only ask about ones that would run on the equipment you have.

How does this product work?

Ask the vendor how the product is designed to work. Although technology tools should be somewhat intuitive, approaching a new product with no direction may waste valuable time. Ask the vendor for a product demonstration. During a demonstration, you can usually get an idea about "undeclared" requirements, for example, that the product has password-protected entry tied to licensed users or that it requires additional equipment or other products to get the advertised results. Be sure to ask what else is needed to get the product up and running. It may be as simple as requiring an active Internet connection that is not readily accessible in your educational setting.

How do the product's educational objectives correlate with the educational goals of the school or district?

Even though the content may match curriculum goals, the way students access the content must also fit with the educational goals of the school or district.

How steep is the learning curve?

Products that require schoolwide training workshops will typically require district approval. To get this, you must be able to estimate the amount of training staff will need before effectively using the tools with students and then find the funding to successfully implement the training.

What research supports the teaching pedagogy?

Do not be afraid to use the word pedagogy when questioning vendors. This serves as a quick route past the sales force and on to the product developers, with whom you can discuss the research underlying the product claims. Two types of research should be readily available for any education product: first, the body of research that addresses how students learn best, and second, the research that tested the learning effectiveness of the specific product.

Be prepared to question what exactly is meant when a sales person claims that more than 80 percent of the students advanced one grade level in five months of regular use of the product. Does this mean that four out of five students (all children of employees; the fifth quit when his mom was fired) worked day and night to pass the test? A more meaningful statement of results would be: "80.2 percent of 5,238 students gained one year of growth in reading, practicing 15 minutes a day, three days a week for five months." Now that is a product worth considering.

What exactly does the purchase price buy?

Clarify if the purchase price includes supporting materials, upgrades, and technical support at no additional cost. Supporting material might include a binder containing classroom lessons, pages that students may copy, and lists of supplementary resources. Also important are assessments and the means for correlating student achievement results with state standards. Special Web sites designed for product users to collaborate and grow from each other's experiences are becoming more common. Technical support is crucial to include replacements from breakage, additional downloads when corrupted, or online access to upgrades.

What evidence is there that students successfully use the product?

Ask to see a video of students having success with the product. The environment, student comfort, and teacher actions can provide insights not gathered and published in the research. Consider showing the video or other evidence to administrators and parents and discussing the implications for your school and your students.

Will the vendor demonstrate the use of the product in the classroom or offer a preview period?

Do not commit valued and limited funds to untried new products. Ask for a preview copy of the product and a reasonable amount of time to work with it. Clarify how many computers can be loaded for testing and even ask to install software applications on your home computer during testing and even after the product has been purchased. Confirm compatibility with existing hardware and software and the school district's filtering system.

What are the user's legal rights to the product?

Licensing agreements stipulate the number of computers legally entitled to run software products. Learn whether the teacher has the legal right to install the program at home for planning, or whether duplicating the documentation is considered copyright infringement. There may be regulations about downloads from the company Web site for use in the classroom. Modeling legality in the classroom is not just a matter of economy, it is critical to the classroom environment.

What is the price per seat and the costs for product upgrades?

Many companies have two separate channels for selling software--commercial and educational. The educational reseller may be a separate division or separate company that focuses on educational software and education markets. It is of no consequence how the vendor's business is structured, but it is important to seriously consider whether the vendor takes purchase orders, whether a parents' group can buy at the discount rate, and whether the purchase has to be submitted on school letterhead and come from the school budget. Pricing for the education market varies greatly from the pricing for the commercial market. An education price may look expensive when compared to prices on some of the economy Web sites, but consider how much is included in the purchase price--binder, upgrades, replacements, additional CDs at low cost, etc. Businesses do not receive the same price breaks as education, so being responsible examiners and not copying software can help us retain these hard-fought-for opportunities.

Resources

Three consortia that publish their results on the Internet provide solid background information about evaluating technology resources. All are limited by budgetary constraints, but all seem driven by the need to help teachers make quality choices to advance effective technology integration.

• Multimedia Educational Resource for Learning and Online Teaching (MERLOT) is designed to identify and share information about materials for higher education. More than 4,000 of almost 16,000 members are categorized under Education. Go to http://www.merlot.org. Join for free, suggest materials to examine, and add your evaluations to the products listed.
• EvaluTech provides valuable resource reviews and evaluation guidelines through its Web site at http://www.evalutech.sreb.org. The state departments of education of Alabama, Arkansas, Delaware, Florida, Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, Oklahoma, South Carolina, Tennessee, Texas, Virginia, and West Virginia collaborate with the North Carolina Department of Public Instruction to assist educators to identify and integrate quality technology resources. Each state assigns a contact to work directly with the North Carolina Department of Public Instruction to keep EvaluTech up-to-date and useful.
• Educational Software Preview Guide Consortium (ESPG Consortium) has been hindered in some ways by its broad-based membership but enhanced by its variety. The Consortium is not funded through a grant or supported by any state department of education, but it is long-standing and currently publishes its results through one of its members, the Fermi Lederman Science Center in Batavia, Illinois. The ESPG was published in print format from 1983 through 2001, first by the state of California and since 1989 by ISTE (International Society for Technology in Education). Although the guide is no longer sold in print format, it is still available through numerous Web sites. Search the Guide for free at http://eddata.fnal.gov/lasso/trc_software_review/searchespg/search.html.

And last, but not least, every educator should know about Universal Design for Learning (UDL), which is a powerful approach to teaching, learning, and assessment developed by the Center for Applied Special Technology (CAST). CAST is a not-for-profit, education research and development organization that works to create opportunities for all students, especially those with learning disabilities, physical disabilities, and other challenges, by using technology to make education more flexible and accessible. The site examines existing software and instructional practice through the UDL lens. And, as a result of CAST's R&D, thousands of classrooms now use leading-edge, UDL-based products, including: Thinking Reader, WiggleWorks, Bobby, and CAST eReader. Visit http://www.cast.org and discover educational methods and materials that are flexible and powerful enough to maximize the progress of all students, regardless of their ability.

Ready, Set, Go!

Teachers, your students are waiting, your administrators are demanding, and parents are on the phone. Check out these resources for reliable technology tools and save yourself some time.

Judi Mathis Johnson, Ph.D., is on the faculty at Lesley University. She continues to teach educational software evaluation as well as other courses for the Technology in Education Masters Program where students focus their critical skills on effectively using technology resources in education. She was Software Editor for ISTE for 15 years.

back to top

Home   Past Issues   Editorial Guidelines   Submission Requirements

Northwest Educational Technology Consortium http://www.netc.org