Quantum Tutoring Software Improves Student Test Scores By Full Letter Grade

February 2, 2005

Pittsburgh, Pennsylvania – February 1, 2005, Quantum Simulations, Inc., a developer of artificial intelligence (AI) tutoring and assessment software, announced the results of a research study evaluating the effectiveness of Quantum’s Chemistry Tutors and Automatic Grading/Assessment Tool at Carmichaels Area Junior/Senior High School in Carmichaels, Greene County, Pennsylvania.

The one-week study included 51 high school students from Carmichaels Area Junior/Senior High School. Students were divided into two groups:

    -Treatment group (31 students), who used the Quantum Tutors for at-home study.
    -Control group (20 students), who received the same classroom instruction as the Treatment group but did not use the Quantum Tutors.
Both the treatment and control groups were given pre- and post-tests to benchmark and evaluate performance in understanding balancing chemical equations, an essential fundamental topic covered in high school chemistry.

The results demonstrated that students using the Quantum Tutors as part of their at-home study outperformed the control group on the post-test by an average of 12.8 points, or just over a full letter grade (see chart below). In addition, the difference in test scores between the treatment and control groups widened as the problems became more difficult to solve – a clear indication that understanding and comprehension significantly improved for students using the Quantum Tutors.


Student Performance: Pre- and Post-Test Results Carmichaels Area Junior/Senior High School


Treatment Group (Used Tutors) Control Group
Pre-Test Score 53.9 53.6
Post-Test Score 74.3 61.5
Improvement +20.4 +7.9

  • Out of a maximum of 100 points.
    “I certainly wasn’t a skeptic, but I must admit, I was surprised with the results,” commented Kevin Willis, Chemistry Teacher and research participant at Carmichaels Area Junior/Senior High School. “The group of students using the Tutor drastically improved performance even with the most difficult problems.”

    “The clear differences in the various trends in the treatment and control group’s performance observed in these studies at the high school level indicate that the Quantum Tutor improved student ability to balance chemical equations, further validating the AI-based approach Quantum has developed,” added Selma Sax, Senior Evaluator at the Wexford Institute, a nonprofit educational agency evaluating the Quantum Tutors and Automatic Grading/Assessment Tool.

    To provide an accurate assessment of student knowledge and comprehension, all student work in the study at Carmichaels was graded using a new automatic grading/assessment technology developed by Quantum. Utilizing Quantum’s patented artificial intelligence technology, the tool evaluates every step in the student’s work and assigns partial credit, rather than simply grading whether an answer is right or wrong as with multiple-choice tests.

    Mr. Willis used the new assessment technology during the research and commented, “The automatic grading/assessment tool is able to gather more information about a student’s actual understanding of a concept than traditional by-hand grading, not to mention, it is more accurate! This feedback is important to the teacher and student alike.”

    According to validation studies, the AI grader has been established as significantly more accurate and reliable than teachers grading by hand.

    For the student, the grading/assessment software automatically grades the student’s work and provides immediate, detailed feedback as well as tutoring help on topics and concepts needed to improve future test scores. For the teacher, the software eliminates the administrative time and human error and fatigue associated with grading papers, while providing teachers with real-time data for analyzing how the class performed on the concepts tested with drill-down detail to each individual student’s work. This on-demand assessment is particularly useful as states struggle with increasing graduation requirements in science, standards-based instruction and high-stakes testing.

    The research studies conducted at Carmichaels Area Junior/Senior High School in western Pennsylvania is part of a national research effort in collaboration with the Wexford Institute, a nationally recognized nonprofit agency dedicated to creating equitable and excellent educational opportunities for all learners, to measure the effectiveness of the Quantum Tutors and new automated AI-based grading/assessment technology in preparation for the No Child Left Behind (NCLB) science assessments in 2007. Research funding for these studies is provided by the U.S. Department of Education and the National Science Foundation.

    ABOUT QUANTUM
    Quantum Simulations, Inc. develops artificial intelligence (AI) tutoring, assessment and professional development software that empowers teachers and inspires students from middle school through college to improve their knowledge and appreciation for the sciences. Teacher tested and approved by educators across the United States, Quantum’s AI software is proven in research studies to improve comprehension, problem solving skills and test scores by as much as 50%. Quantum’s intelligent tutoring engines are integrated with existing web-based learning products, providing a strong competitive edge for distributing partners.

    A “technology think tank”, Quantum is funded and supported by the U.S. Department of Education, the National Science Foundation and the National Institutes of Health. For a virtual demonstration of the Quantum Tutors, visit www.quantumsimulations.com/demo.html.

    For a list of distributing partners visit www.quantumsimulations.com/partners.html


    For complete published research results, contact Dr. Benny Johnson at johnson@quantumsimulations.com.
    Al Renshaw
    Director, Marketing and Sales
    Quantum Simulations, Inc.
    5275 Sardis Road
    Murrysville, PA 15668
    Phone: (724) 772-8014
    ax: (724) 772-8042
    http://www.quantumsimulations.com