SBIR Phase I: Grading 10x Faster with AI Assistance

SBIR Phase I: Grading 10x Faster with AI Assistance

Award Information
Agency: National Science Foundation
Branch: N/A
Contract: 1747365
Agency Tracking Number: 1747365
Amount: $224,700.00
Phase: Phase I
Program: SBIR
Awards Year: 2018
Solicitation Year: 2017
Solicitation Topic Code: EA
Solicitation Number: N/A
Small Business Information
2030 Addison St Suite 500, Berkeley, CA, 94704
DUNS: 079771662
HUBZone Owned: N
Woman Owned: N
Socially and Economically Disadvantaged: N
Principal Investigator
 Sergey Karayev
 (425) 218-9416
 sergeyk@gradescope.com
Business Contact
 Sergey Karayev
Phone: (425) 218-9416
Email: sergeyk@gradescope.com
Research Institution
N/A
Abstract
This SBIR Phase I project will show feasibility of a method for instant and accurate grading of student answers to a previously unseen question, after observing an instructor grade no more than 10% of the answers to that question. This project will use instructor-defined grading rubrics, which ensure consistency and provide helpful feedback to the student. Initial work will evaluate the method on multiple choice and fill-in-the-blank questions from a variety of Science, Technology, Engineering, and Math (STEM) subjects. The same method will be general enough to allow extension to short-answer and diagram questions. This work is important because although constructed-response questions are far more effective at assessing student knowledge and guiding student learning, they are significantly more time-consuming and onerous to grade than multiple-choice questions, and are therefore underused in many courses. Giving instructors the ability to grade constructed-response questions 10 times faster will increase the prevalence of this type of assessment and thereby improve STEM education outcomes. It will also increase educator effectiveness, because time that instructors currently spend grading can be better spent in personalized interaction with students. This project will apply and extend recently developed deep neural network methods for few-shot learning to the task of using a small sample of graded student answers to automatically grade the rest. Deep neural networks have recently emerged as the best approach to most machine learning problems. However, unlike humans, who can often learn a new concept after just a few examples, deep neural networks typically need thousands of examples to do the same. Recent developments have enabled deep neural networks to learn new concepts from just a few examples, after seeing millions of examples of other concepts. The first objective of this project is to evaluate these methods using millions of answers to a range of multiple-choice and fill-in-the-blank questions from a diverse set of STEM subjects graded on an existing service. This few-shot learning problem is significantly more complex than problems studied in the research literature. The second objective is to extend these methods, and to develop new methods, in order to reach human-level accuracy after observing the grading of no more than 10% of student answers. The last objective is to ensure that the proposed system is useful to real instructors by implementing a prototype user interface.

* Information listed above is at the time of submission. *

Agency Micro-sites

SBA logo
Department of Agriculture logo
Department of Commerce logo
Department of Defense logo
Department of Education logo
Department of Energy logo
Department of Health and Human Services logo
Department of Homeland Security logo
Department of Transportation logo
Environmental Protection Agency logo
National Aeronautics and Space Administration logo
National Science Foundation logo
US Flag An Official Website of the United States Government