Resource Updates 2023-01-05
Discover more from Sentient Syllabus
Resource Updates 2023-01-05
Update on updates
This letter is a summary of recent updates to the Sentient Syllabus resources.1
General
The discussion is very much in flux. We have not yet seen official institutional positions, though we hear they are in the works. Adding a date-of-last-revision to the sections of the resource files was good advice. Thank you. The date is always to the right of the major section headers.
Syllabus Resources
Suggestion and sample text to add an “in-flux” disclaimer to syllabi.
Some clean up of style and ambiguities.
Course Activities
New section on “Critique and Improvement”. The idea that there is much opportunity to learn from a critique of AI contributed text is gaining traction. Two points merit attention. (A) We need to remind ourselves that the target is not the AI, but the subject matter. (B) The AI itself (obviously) can write a critique of its own writing, and suggest improvement. Such a critique is not proof that a human mind was involved. We’ll post on that topic later this week. Criteria for assessing “good” critique focus on content that is specific to the subject, and that the AI is (still) poor at. This includes specific facts, precise attribution,2 and anything really that requires actual analysis of the text, such as assessing logical coherence, assessing whether arguments are redundant, assessing whether arguments are similarly important. That is a natural consequence of the way these kind of language models work. A table of seven criteria is presented, but getting this to make sense to students will need examples.
New section on “Personalized Tutoring”. This is a great opportunity, if we can figure out how to make this work in practice. The AI is patient, the AI is knowledgeable, the AI writes very well, the AI can make hilarious mistakes. Learn to be cautious. Here too, a tempering thought: tools are worthless if they are not adopted; in in my experience, it is often those students who need the most help that are the least likely to engage with it. There may be sensitive cultural issues involved. Creative incentives may help, practice will teach us more.
Learning Objectives
No significant updates.
Understanding AI Issues
How does it work? Brief, non-technical synopsis of the training data, and training process towards an intuition how such “Large Language Models” work in principle. Note that the machine does not “understand” its output in the way we understand our speech, but the understanding is implicit in the understanding of the authors of the source data. (There are philosophical implications to this …). Introduction of the term perplexity which is a measure of how unexpected an output is. Training the AI for low-perplexity is a source of its impeccable grammar, and spelling – and its bland, insipid style.
Plagiarism: major update – plagiarism detection tools will fail, and that is also true for the latest hype of an app that uses “perplexity and burstiness” to identify ChatGPT as an author. Three points to consider: (A) Plagiarism detection operates in an adversarial context, i.e. text will be manipulated to fool the detectors. Humans will find a way. (B) The overlap for any given feature between human and synthetic text is large. Of course, that is what the AI was designed to do. (C) The false positive rate is high. This, we cannot tolerate. The false-positive rate for plagiarism needs to be indistinguishable from zero in the context of an academic institutional policy. We are expressing a counterposition: unrestricted access. Time will tell whether this can be made practical, and acceptable in the chancelleries and hallways.
Perspectives: Thoughts and background that will help cultivate our intuitions: energy balance; the prospects of personal AI systems; perspectives on more data e.g. translations; perspectives on better data (e.g. books); a brief perspective on machines-training-machines and exponential acceleration.
Enjoy, and … feedback, comments, and experience are welcome – sentient.syllabus@gmail.com
Cite: Steipe, Boris (2023) “Resource Updates 2023-01-05”. Sentient Syllabus 2023-01-05 https://sentientsyllabus.substack.com/p/resource-updates-2023-01-05 .
See “Chat GPT’s Achilles Heel”, here.