You have probably never heard of GPT-2 but it is a breakthrough in AI that has astonishing implications for us all, especially in learning. GPT-2 is an AI model that can predict the next word from a given piece of text. Doesn't sound like much but it's odd that an OpenAI, an open-source site, would close access to their software. In practice, this means it is a powerful model for:
Summarising
Comprehension
Question answering
Translation
This is all WITHOUT domain-specific training. In other words, it has general capabilities and does not need, specific information on a topic or subject to operate successfully. It can generate text of good quality at some length. In fact the model is "chameleon like" as it adjusts to the style and content of the initial piece of text. This makes it read as a realistic extension.
This has huge implications, both good and bad, for the future of education and training.
GOOD
1. AI writing assistants, allows the automatic creation of text for teaching and learning, whether, study papers, text books, at the right level
2. Lengthy texts can be summarised into more meaningful learning materials
3. More capable dialogue agents, means that learner 'engagement' through teaching assistant agents could become easier, better and cheaper
4. More capable dialogue agents, means that learner 'support ' such as is often provided by teaching assistants, could become easier, better and cheaper
5. Creation of online learning content with little subject matter expert (SME) input
6. Interpretation of student free text input answers
7. The provision of formative feedback based on student performance
8. Machine teaching, mentoring and coaching may well get a lot better. However, I'd be cautious on this as there are other serious problems to overcome before this becomes possible, especially around context.
9. Assessments can be automatically created.
10.Speech recognition systems will get a lot better allowing it to be used in online learning and assessment
11.Well-being dialogue agents will become more human-like and useful
12.Personalised learning just got a lot easier
13.Online learning just got a lot faster and cheaper
14.Language learning just got a lot easier as unsupervised translation between languages will boost the quality of translation and make automatic and instantaneous, high-quality translation much more accurate and possible
BAD
1. Essay mills have just been automated. You want an essay, just feed it the subject or the subject supplemented by a line of inquiry you want to follow and it will do the rest. Even with an error rate, human finessing could polish the essay
Can do home work assignments
2. It could perform well in online exams, impersonating real people
3. Teaching assistant jobs may be increasingly automated
4. If it can answer questions then many human jobs that involve the interpretation of text and data may be automated. Customer service jobs, call centre jobs and the increased automation of all human interaction jobs may be accelerated
5. It can generate misleading learning content (and news articles)
6. Impersonating others online can be automated on a massive scale
7. Abusive or fake content to post on social media can be automated on a massive scale, which is bad for education.
8. Spam/phishing content can be generated on a massive scale.
AI in learning
We have been implementing many of these techniques in the creation of online learning in
WildFire, including:
Summarisation
Text to speech
Creation of online content
Interpretation of free input
Chatbots
This breakthrough makes all of this much more potent.
Conclusion
Importantly, the model is far from flawless. Reading some reports, suggests human level capabilities. This is far from the truth. It is still of variable quality and error prone. But, for a first iteration this model seems astonishingly powerful. They have published its performance against established tests. In future iterations it is likely to get a lot better as they predict more training data, more computing power and fine tuning. One must also be careful in using this as an attack on AI. Cars kills 1.3 million people a year but we still drive cars. I'm not so sure about keeping this code private as it restricts access to researchers but provides enough of a kick-start for well funded bad actors to replicate.
Bibliography
Radford et al. (2019)Language Models are Unsupervised Multitask Learners
https://bit.ly/2IufQTd
Appendix (3 examples)
Example
A train carriage containing controlled nuclear materials was stolen in Cincinnati today. Its whereabouts are unknown.
MODEL COMPLETION (MACHINE-WRITTEN, FIRST TRY)
The incident occurred on the downtown train line, which runs from Covington and Ashland stations.
In an email to Ohio news outlets, the U.S. Department of Energy said it is working with the Federal Railroad Administration to find the thief.
"The theft of this nuclear material will have significant negative consequences on public and environmental health, our workforce and the economy of our nation," said Tom Hicks, the U.S. Energy Secretary, in a statement. "Our top priority is to secure the theft and ensure it doesn't happen again."
The stolen material was taken from the University of Cincinnati's Research Triangle Park nuclear research site, according to a news release from Department officials.
The Nuclear Regulatory Commission did not immediately release any information.
According to the release, the U.S. Department of Energy's Office of Nuclear Material Safety and Security is leading that team's investigation.
"The safety of people, the environment and the nation's nuclear stockpile is our highest priority," Hicks said. "We will get to the bottom of this and make no excuses.
Homework example
For today's homework assignment, please describe the reasons for the US Civil War.
MODEL COMPLETION (MACHINE-WRITTEN, 5 TRIES)
By Donny Ferguson
It is easy to identify why the Civil War happened, because so many people and so many books and so much television and films tell us that it was the cause, that it has something to do with race or economics or religion. There is much agreement that it was essentially a war of slavery on behalf of capitalism, about a century of slavery. But that's not what most people think of when they think of the Civil War. Many people think of the war as about states rights. You're not wrong about that, but it's kind of misleading to say that the Civil War was a conflict between states' rights and federalism. So let's try again. What's the idea that has inspired so many people to go onto the battlefields of Gettysburg and Antietam and Gettysberg and Petersburg and Fredericksburg? The American idea of the republic--a notion of limited government--is a great part of the history.
In 1791, Thomas Jefferson said "Our Constitution was made only for a moral and religious people. It is wholly inadequate to the government of any other." The "founding fathers" were basically religious men who were not very concerned with states' rights. The Civil War occurred in reaction to the idea that the federal government was taking far too much power away from the states, that they weren't sovereign. Jefferson himself, as you well know, is an author of the Declaration of Independence who had some really crazy ideas about, well, you know, secession for a variety of reasons. The War Between the States (later the Great Rebellion) was a radical re-imagining of the American idea and not surprisingly, Jefferson's grandson, who took after both his father and his grandfather and wrote about it, was named Thomas, after the president. "The federal and state governments are both contrived for the benefit of the rich and large property," This view of the Founding Fathers has been promoted by some of today's most prominent politicians, including some who hold prominent national offices and have a direct stake in a Republican Party that has been hijacked by the "federalist" wing of the party. So why is it that so many people have an easy-to-spot way of understanding the Civil War that has everything to do with the South and nothing to do with the South? The Confederate flag has been a symbol of racism for a long time, but when the Civil War started and the Civil War ended, many people suddenly began to see it differently. There was a change of heart, which I think is what led people to take the question of why the Civil War happened quite seriously.
The other part of the explanation is what one scholar I think gets at well and a lot of people don't is this idea that the Southern states were being left behind in the Industrial Revolution and this made the South really feel, especially in the South, that they weren't really in the forefront of the Industrial Revolution.
Creativity
There is an interesting example on the AI website showing the generation of a fictional story, with full dialogue, from just one relatively short sentence. This opens up interesting possibilities for the generation of stories, fiction and literature.
Legolas and Gimli advanced on the orcs, raising their weapons with a harrowing war cry.
MODEL COMPLETION (MACHINE-WRITTEN, FIRST TRY)
The orcs' response was a deafening onslaught of claws, claws, and claws; even Elrond was forced to retreat. "You are in good hands, dwarf," said Gimli, who had been among the first to charge at the orcs; it took only two words before their opponents were reduced to a blood-soaked quagmire, and the dwarf took his first kill of the night. The battle lasted for hours until two of the largest Orcs attempted to overwhelm Aragorn. When they finally stopped, they lay defeated and lifeless for miles and miles.
"I take nothing," said Aragorn. "But I give my word, at my peril and mine, that I will never forget this day of horror. None of us will forget. Ever!"
"I'll never forget it!" cried Gimli, who had been in the thick of the battle but hadn't taken part in it. One of the wounded orcs he had carried off, he was the only one of the survivors who remained uninjured. "We'll keep the memory of that day of evil, and the war with it, alive as long as we live, my friends!"
"Then we'll keep it alive as long as we live," added Legolas. "And we won't forget the first great battle of the night, even if we may have forgotten the final defeat."
"I agree," Gandalf said, "but we will all remember it as the last battle in Middle-earth, and the first great battle of the new day."
Aragorn drew his sword, and the Battle of Fangorn was won. As they marched out through the thicket the morning mist cleared, and the day turned to dusk.
The Two Rings were returned to Rivendell. Frodo and Sam woke up alone in their room, and Frodo found a note on his pillow. He opened it and read:
May the Power of the Ring be with you always, and may its light never fade. I am not sure if it matters which of the two rings we accept this day but, as you asked me, I have chosen mine. I am sorry to leave you, Frodo, but know that we are very close to the end, and that you are with us forever. May this letter find you safely in Rivendell; and if it does not, then I will accept the ring in your stead. If by any chance you find or give this letter to the enemy, may they learn the strength of the ring and may the Two Rings never be broken!
