1

Moving Beyond the Detection Arms Race

2

The Current Landscape

How Students Use AI for Academic Dishonesty

3

Current Detection Methods

Pattern Recognition analyzes linguistic patterns that AI models tend to produce. This includes:

  • Repetitive sentence structures
  • Overuse of certain transitional phrases
  • Consistent paragraph lengths
  • Formal tone throughout

Limitation: Students can easily edit out these patterns with minimal effort.

Statistical Analysis examines the probability of word choices and combinations:

  • Frequency of rare vs. common words
  • N-gram analysis (sequences of words)
  • Vocabulary sophistication scores
  • Syntactic complexity measures

Limitation: AI models are becoming increasingly sophisticated at mimicking human statistical patterns.

Perplexity Scoring measures how predictable text is to a language model:

  • Low perplexity = predictable, possibly AI-generated
  • High perplexity = surprising, possibly human
  • Based on probability distributions of word sequences

Major Flaw: This assumes AI writes more predictably than humans, but modern AI is trained to be less predictable.

4

Why Detection Will Always Lag Behind

Argument 1: The Proliferation Problem

Premise 1: New AI models are released faster than detection tools can adapt
Premise 2: Each model has unique writing patterns
Premise 3: Detection tools must be retrained for each new model
Conclusion: Detection tools are perpetually outdated
5

Why Detection Will Always Lag Behind

Argument 2: The Customization Problem

Premise 1: AI models can be fine-tuned and prompted to mimic specific styles
Premise 2: Students can train AI on their own previous writing
Premise 3: Personalized AI output is indistinguishable from human writing
Conclusion: Detection becomes impossible when AI perfectly mimics individuals

How Students Can Train AI on Their Own Writing:

  • Upload previous papers to custom GPT models
  • Use few-shot learning with their own examples
  • Fine-tune local models on personal writing samples
  • Create detailed prompts that replicate their style

The Result: AI that writes exactly like the student, making detection nearly impossible.

6

Why Detection Will Always Lag Behind

Argument 3: The Technical Arms Race

Premise 1: AI companies optimize models to sound more human
Premise 2: Detection tools reveal what makes text "detectable"
Premise 3: This information is used to make less detectable AI
Conclusion: Detection research directly enables evasion
7

A New Paradigm: From Detection to Verification

Instead of asking: "Did an AI write this?"

We should ask: "Did this specific student write this?"

Key Shift:

Key Insight: This is the core insight. We have extensive data about each student's authentic writing. Let's use it.
8

Authorship Verification Overview

How It Works

Why We Can Verify Better Than Students Can Fake

What Students Think We Have:

  • Their final papers
  • Maybe some major assignments

What We Actually Have:

  • Every discussion post
  • In-class timed writings
  • Email communications
  • Group project contributions
  • Early drafts and revisions
  • Peer review comments
  • Office hours conversations (noted)
  • 4+ years of progression

The Key: Authentic writing in low-stakes, unguarded moments

Think of it as: Handwriting analysis for the digital age, but far more sophisticated and privacy-protected.
9

System Design Overview

Student Data → [SECURE ANALYSIS BOX] → Probability Score

↓
No data visible
Only outputs:
- Match probability
- General inconsistency flags

Privacy Guarantees:

Input Vectors:

  • Stylometric features (syntax trees, n-grams)
  • Semantic embeddings (BERT/GPT encodings)
  • Demographic priors (regional language models)
  • Temporal progression models

Processing:

  • Ensemble ML classifiers
  • Bayesian probability networks
  • Anomaly detection algorithms

Output:

  • P(authorship | evidence)
  • Interpretable inconsistency flags
10

Advantages Over Detection

Why This Approach Wins

11

Adoption Strategy

Incentivizing Universities Through Student Choice

The Opt-In Verification System

Beginning of Semester: Students choose whether to have their authorship verified for the class
Same Grade Regardless: Verification choice does not affect academic assessment
Verified Work Recognition: Students who complete verification receive an additional credential marker
Opt-Out Option: Students can withdraw from verification at any point before grades are finalized

The Credential Advantage

For Students

Verified ✓ A > Regular A
Verified ✓ B > Regular B

Enhanced employability

For Employers

Clear indicator of authentic academic work

Confidence in candidate abilities

For Universities

Competitive advantage in graduate/employer reputation

Academic integrity leadership

Market Dynamics

Market Dynamic: This creates a positive feedback loop where verification becomes valuable without being mandatory, driving adoption through market forces rather than institutional mandate.
12

Addressing Concerns

Potential Objections & Responses

"This is surveillance"

  • No more than current plagiarism detection
  • Data never displayed or shared
  • Students maintain full privacy

"Students' writing naturally evolves"

  • System accounts for growth
  • Flags only dramatic inconsistencies
  • Human review always final

"What about ESL students?"

13

A Pedagogical Innovation

New Assignment Type: "Prompt Engineering for Perfect Essays"

The Assignment:

Students must create the perfect prompt to generate the perfect essay

They are evaluated on both the resulting essay AND the prompt itself

What Students Must Learn:

Required Components:

📚 View Live Example: The Euthyphro Dilemma Assignment

Example Scenario - History Class:

Traditional Assignment: "Write an essay on the causes of WWI"

New Assignment: "Create a prompt that will generate an A-level essay analyzing the interconnected causes of WWI, demonstrating understanding of historiographical debates, primary source integration, and causal complexity"

Students must then provide historical context, define historiographical schools of thought, give examples of strong arguments, and specify citation formats - all requiring deep subject mastery.

14

Call to Action

Next Steps

The Question: Will we keep playing catch-up, or lead the way?

16

Questions & Discussion

Contact:

📧 mgallagh@gmail.com (personal)

📧 mgallagher@demadtech.com (work)

📱 (310) 966-7838


"The best way to predict the future is to invent it." - Alan Kay

Actually I don't know if this quote is accurate but it sounds good.
A

Appendix

Technical Terms Glossary

N-grams

Sequences of N words that appear together. Example: "to be or" is a 3-gram. Used to analyze writing patterns.

Token

A unit of text (usually a word or part of a word). "Understanding" might be 3 tokens: "Under" + "stand" + "ing"

Perplexity

How surprising or unpredictable text is. Low = predictable/boring, High = creative/unusual

Burstiness

Variation in sentence length and complexity. Humans write in "bursts" - short then long sentences. AI tends to be more uniform.

Stylometry

The study of linguistic style. Like a fingerprint for writing. Analyzes word choice, sentence structure, punctuation.

Semantic Embedding

Converting meaning into numbers. Allows computers to understand "dog" is similar to "puppy". Powers modern AI understanding.

Ensemble Methods

Using multiple algorithms together. Like getting second opinions from multiple doctors. More reliable than any single method.

Bayesian Probability

Updating beliefs with new evidence. Start with what we know about a student. Adjust confidence as we see new writing.