Can A.I. Grade Your Next Test?
This decade of data is at the origin of the new experiment of the university in artificial intelligence.
Dr Finn and his team have built a neural network, a mathematical system that can learn skills from vast amounts of data. By spotting patterns in thousands of cat photos, a neural network can learn to identify a cat. By analyzing hundreds of old phone calls, he can learn to recognize spoken words. Or, by examining how teaching assistants assess coding tests, he can learn to assess those tests on his own.
Stanford’s system has spent hours analyzing examples from past semesters, learning from a decade of possibilities. Then he was ready to find out more. When given just a handful of more examples of the new review proposed this spring, he could quickly grasp the task at hand.
“He sees many types of problems,” said Mike Wu, another researcher who worked on the project. “Then he can adapt to problems he’s never seen before.”
This spring, the system provided 16,000 comments, and students agreed with the comments 97.9% of the time, according to a study by researchers at Stanford. In comparison, students agreed with comments from human instructors 96.7% of the time.
Mr Pham, an engineering student at Lund University in Sweden, was surprised that the technology works so well. While the automated tool was unable to evaluate any of its programs (likely because it had written a snippet of code unlike anything the AI had ever seen), it both identified bugs. specific in its code, including what is known in computer programming and mathematics as a fence post error and suggested ways to correct it. “It’s rare that you get such thoughtful feedback,” Pham said.
Technology was effective because its role was well defined. While taking the test, Mr. Pham wrote code with very specific purposes, and there were only a number of ways he and other students could go wrong.
But with the right data, neural networks can learn a whole range of tasks. It’s the same foundational technology that identifies faces in photos you post to Facebook, recognizes commands you bark on your iPhone, and translates between languages on services like Skype and Google Translate. For the Stanford team and other researchers, the hope is that these techniques can automate education in many other ways.