More than 20,000 software programs claim to provide educational value. Some titles soar. Unfortunately, more of them stink. You need a way to find out which is which? But you certainly can’t rely solely on the description on the package or in the catalog.
When you’re ready to buy classroom and curriculum planning software, you need a thoughtful purchasing strategy. You want to know accepted criteria for evaluating software. You should find out how your colleagues rate the companies selling software (see eSchool News Buyer’s Guide Five Star Rating System: http://www.eschoolnews.org/buyersguide) and what they have to say about specific products.
In this article, you’ll review basic strategies for evaluating software and then find links to some of the nation’s leading sources of educational software reviews.
To establish a responsible software selection process, it’s recommended you follow a step-by-step approach, something along the lines of this systematic strategy, based on “Seven Steps to Responsible Software Selection” a public-domain document available at AskERIC and written by Kenneth P. Komoski, of the Educational Products Information Exchange (EPIE), and Eric Plotnick, of the Information Institute of Syracuse:
Step 1. Analyze Needs
First determine whether or not the computer is the appropriate medium to use to satisfy your specific instructional goals and objectives. The possibility always exists that a careful needs analysis will result in a decision to use some other learning strategy. Decide on the most appropriate medium by weighing your needs, goals, and objectives.
Needs & Goals. A need is the difference between “where we are now” (e.g. 60% of the students in the ninth grade score above minimum competence on the state science test) and “where we would like to be” (e.g. 90% of the students in ninth grade score above minimum competence on the state science test). “Where we would like to be” is another way of defining a goal.
Objectives. An objective describes “where we would like to be” in more specific terms (e.g. 90% of all ninth grade students will exceed the minimum level of competence on the state competency test administered in the second semester of ninth grade). Objectives must include conditions under which the desired behavior will be demonstrated and the criteria for measuring that behavior.
Educational objectives help us respond to needs by breaking them down into attainable steps, making it easier to get from “where we are now” to “where we would like to be.” The educational objective stated above is a “terminal” objective which must be broken down into a series of “enabling” objectives (e.g. By October 31, 2003, all ninth grade students will be able to correctly identify at least five out of seven minerals when shown them by the teacher.) Enabling objectives identify specifically what behavior we would like the student to demonstrate. For each enabling objective, you should brainstorm alternative learning methods for achieving that objective, such as direct student teacher interaction, self-instruction workbook, videotape, computer-assisted instruction, and so on.
After considering the benefits and constraints of each learning method, you should be able to make an informed decision about which medium or combination of media will satisfy the identified needs, goals, and objectives.
Step 2. Specify Requirements
If a careful needs analysis determines that computer assisted instruction is one of the methods that will be used to meet identified instructional objectives, you should then specify the requirements for the computer software. Factors to consider in specifying requirements for software include: compatibility with available hardware; cost (Will the school need multiple copies of the software? Will a site license be necessary? ); user friendliness; level of interaction desired; adequacy of documentation; access to technical support via toll-free number; and of course, direct correlation with the instructional objectives and curriculum requirements identified in the needs analysis.
Researchers have suggested you should apply the following criteria within the context of your objectives and the students’ needs: content, instructional presentation, demands placed on the learner, technical features, documentation, and management features.
Step 3. Identify Promising Software
If software requirements are specified in detail, you will have a good head start when it comes to identifying promising products. There are many ways to identify promising software, and you should use as many of them as possible. Catalogs still remain an important source for descriptions of software. Most district-level educational communications/media centers are on catalog mailing lists from virtually all software producers and wholesalers. Software is advertised, described, and often reviewed in the magazines, newspapers, and journals found in school, university, and public libraries. The Educational Products Information Exchange produces The Educational Software Selector (TESS; http://www.epie.org/epie_tess.htm), a for-fee database containing descriptions and reviews of thousands of currently published educational software programs.
Such information is also available in free databases, several of which are listed at the end of this article. Another way to find out about software from other educators by joining a listserv or discussion forum.
Posting a question such as, “I am an eighth grade science teacher, and I am looking for interactive software for a PC environment that will teach my students how to . . . ” is likely to bring dozens of responses.
eSchool News provides the eSchool News Forums, where several discussion topics lend themselves to seeking peer counseling on specific software needs. Many listservs are archived at AskERIC
Regardless of how you gather your information, the more precisely the requirements are specified in Step 2, the easier it will be to screen out those products that are least likely to meet your specifications and the easier it will be to focus on more promising products.
Step 4. Read Relevant Reviews
After you identify a list of promising software titles, you may be able to narrow or expand the list by reading relevant software reviews. But this is important: Reading reviews should not take the place of previewing the software, as described in Step 5. Software reviews may be found in educational journals, some of which may be identified by searching the ERIC database using appropriate descriptors (e.g. software, selection, evaluation, elementary, secondary).
Evaluation services such as EPIE, subscribed to by many school and public libraries, provide databases of selected software evaluations and reviews. A visit to the library is an important part of responsible software selection.
Keep Step 1 (Analyze Needs) and Step 2 (Specify Requirements) in mind as you read the reviews. Be sure to note the audience to whom any given review is addressed. A software program might get a poor review because it was tested with a different audience than the one you have in mind. When used as one part of the entire selection process, reviews are important screening tools.
Step 5. Preview Software
The most effective way to judge whether software is appropriate or not is to observe students as they interact with the program. Are the educational objectives achieved when the student uses the program? You shouldn’t purchase software without previewing it with actual students. Preview as many programs as you can find that appear to meet your selection criteria. Some software vendors will allow free preview of an entire program. Some vendors will provide a free demonstration disk containing a subset of a larger program. Some vendors will not allow you to preview their products without a purchase order, but will allow you to return the program within a specified time limit with no financial obligation. In some situations, you might be able to borrow a program from another educator for preview purposes.
As a general rule, if there is no way to preview software under actual conditions-avoid that software.
Step 6. Make Recommendations, complete purchase
After you’ve previewed potential software, it’s time to recommend or complete the purchase. At this time, you should be able to:
•select the most desirable software after a systematic evaluation of all alternatives in terms of educational objectives and constraints;
•establish a quantitative method for rating each alternative against the selection criteria established in Step 2;
•evaluate the relative importance of each selection criterion, (ie. previewing should probably be rated relatively high in importance); and
•create a written record outlining the reasons a specific piece of software should or should not be purchased.
For software selected for purchase, you should include suggestions for optimal use that might have become apparent during the preview period. The written record, including the quantitative rating scale and the selection criteria, should be kept on file for future reference.
Step 7. Get Post-Use Feedback
After software is purchased and used with students, it is important to determine the conformance or discrepancy between all of the enabling objectives specified in Step 1 and the student performance actually obtained using the chosen computer software. Those deploying the software should keep records on the relative extent to which each objective is met or not met. Objectives not met might be addressed by some other software program or by another teaching/learning method. Post-use feedback can be a significant help to a school’s systematic process of software selection, purchase, and use.
The accumulation of user feedback, including anecdotal experience on the part of both teachers and students, will naturally serve to improve future needs analyses (Step 1) and all succeeding steps in a constantly improving software selection process.
Eight Characteristics of Top-Quality Educational Software
(Abstracted from “The Good, The Bad, and the Useless” by Patricia Brogan Electronic School, March 2001)
Educational software is proliferating, and its producers work hard to entice both teachers and parents. Most software packages are drills that help students memorize information or learn a skill. Other software attempts to adapt to a student’s interests and performance, though this type of software is still in the early stages of its evolution.
Here are eight qualities to look for in educational software:
1. Plain and simple interface. Are the key screens well-designed, and can students move from one activity to another? Navigation of the program should be intuitive for learners at the grade level the software is designed for, and icons should be intuitive.
2. Meaningful, but not fancy, graphics. Graphics are only valuable if they support the educational intent. Otherwise, they’re a distraction.
3. Easy exits. Most software contains far more information than a student can process. Make sure it’s easy for the student to exit a specific task-or even the entire program-before frustration sets in.
4. Intelligent interactivity. Drag-and-drop ability and other things that require students to do something can enhance interaction and retention of information greatly.
5. Speed. Students have short attention spans and enjoy fast-paced video games and television shows. Slow educational software will lose them, especially for schools that do not have superfast internet connections.
6. Feedback loops. Good educational software provides some type of feedback to students and teachers that indicates a student’s progress. This information should be in an easy-to-understand format, such as bar graphs. Some software packages also may return the student to information on the topic with which he is struggling.
7. Personalization. Students should be able to log into a system under their own name and retrieve their previous scores. Software also should perform some type of pre-screening of a student’s achievement level, so that subsequent work will be at an appropriate level.
8. Information vs. instruction. Multimedia dictionaries and other reference materials are useful, but they are not educational by themselves. They must be used within a planned curriculum to achieve specific goals. Teachers will need to supply the interactivity to draw out the best use of these types of resources.
Here are several sources of information regarding specific software programs:
2001 Educational Software Preview Guide by Educational Software Preview Guide Consortium & Judi Mathis Johnson, Editor
http://www.iste.org/bookstore/index.html Price $18 for ISTE members; $20 for non-members
http://www.evalutech.sreb.org EvaluTech is a free, searchable database of more than 7,000 reviews of instructional materials, including software, presented through a partnership between the North Carolina Department of Public Instruction and the Southern Regional Education Board. The reviews are written by media specialists, teachers, and other education professionals who reportedly have been trained to use criteria developed by the North Carolina Department of Public Instruction.
Here is an outline of the criteria EvaluTech reviewers use in sizing up instructional software:
CRITERIA FOR EVALUATING COMPUTER COURSEWARE
• Error-free information
• Current information
• Objective, balanced presentation of information
• Bias-free viewpoints and images
• Balanced representations of cultural, ethnic, and racial groups
• Correct use of grammar, spelling, and sentence structure
• Concepts and vocabulary relevant to students’ abilities
• Information relevant to the North Carolina K-12 curriculum
• Interaction compatible with the physical and intellectual maturity of intended audience
• Information of sufficient scope to adequately cover the topic for the intended audience
• Logical progression of topics
• Variety of activities, with options for increasing complexity
Technical Aspects Navigation:
• Rapid retrieval of information and screen transitions
• Intuitive icons, menus, and directional symbols that foster independent use
• Controllable pace, including options for stop/pause/exit
• Controllable sound Save/Record-Keeping Features:
• Options for printing/downloading text
• Save option for games or activities in progress
• Note-taking feature, when appropriate
• Record-keeping feature to monitor student progress Presentation:
• Information presented in a manner to stimulate imagination and curiosity
• Activities that provide opportunities for creative problem solving
• Use of appropriate and supportive feedback
• Options for help, tutorial segments
• Uncluttered screen displays
• Captions, labels, or legends for visuals
• Legible text and print size that is appropriate for the intended audience Quality:
• Visuals relevant to the content
• Sound that is clearly understandable and consistent in quality and volume
• Sound and music that is relevant to screen displays
Documentation Technical Information:
• Descriptions of specific hardware requirements for operating the application
• Instructions for installation and operation
• Toll free technical support telephone number Teacher’s Guide:
• Description of target audience
• Summary of the contents of the application
• Instructional and/or behavioral objectives
• Suggestions for classroom use, lesson plans, related activities
• Ancillary materials for student use, such as camera-ready worksheets and activity pages
Reasonable price in comparison to similar programs.
California Learning Resource Network
CLRN provides educators with a “one-stop” resource for critical information needed for the selection of supplemental electronic learning resources aligned to California’s State Board of Education academic content standards and linked to model lesson plans utilizing technology.
California Educators with specific content experience are selected through an application process to act as reviewers once they have completed a rigorous training program. The review process utilizes the State Board of Education approved review criteria which covers three areas: Legal Compliance, Standards alignment, and Minimum Requirements.
2002-2003 Florida Educational Software Catalog
Each year Florida’s Bureau of Educational Technology surveys every school and district office in the state for nominations of up to 10 educational software titles. These nominations are reviewed by a statewide committee to ensure that the software products will meet the needs of schools throughout the state. This committee then produces the final recommendations that are passed on to the Department for contract negotiations.
At the completion of this process, the Bureau provides access for all Florida public schools to an online catalog containing ordering and contact information for software titles listed in the catalog. These contracts last for one year.
The goal of this project is to allow public schools to acquire instructional software products at a discounted rate. Specific benefits include lower prices for individual schools and districts with limited purchasing volume, and the elimination of internal bid costs for larger school districts. It also provides a benchmark for the pricing of other bids, and provides teachers with information on quality instructional software used by Florida public schools.
eSchool News Readers’ Choice Awards