Good morning. Here is what matters in AI today, and how to put it to work.
We see alignment research fracturing into competing visions while real-world AI misuse in courts signals the governance gap is already here.
~3 min read · last 12 hours
In today's issue
01
Man injected prompts into court filings, suspecting AI was reading them
02
Amazon uses Twitch content to train AI unless streamers opt out
03
Alignment methods may double as a censor's toolkit, researchers warn
04
LLMs can agree with human ethical judgments while reasoning from different moral grounds
05
Position paper: practical alignment must mirror human reasoning, not just outcomes
Main story
Man injected prompts into court filings, suspecting AI was reading them
A litigant, believing the court was using AI to process his filings, embedded adversarial prompt-injection text in documents to try to influence the outcome.
Why it matters: This is the first widely reported case of prompt injection used as a legal tactic, and it exposes a serious gap: if courts use AI tools without disclosure, parties cannot challenge or audit those decisions.
What to watch next: Watch for courts to formalize AI-use disclosure rules in response: if even one jurisdiction mandates it, the precedent will spread fast and reshape how legal teams think about AI-assisted review.
Two stories this week show that AI is already embedded in high-stakes legal and institutional processes, and neither the systems nor the people interacting with them have clear rules for what is allowed.
Oliver Sieberling - Dynamic Short Convolutions Improve Transformers
Cohere
Building Docs for Agents, Not Humans: Inside OpenWiki
LangChain
The Signal
Today's items collectively signal that AI is moving faster than the institutions meant to govern it. Courts may already be using AI to process filings without disclosing it, alignment researchers are warning that the tools built to make AI safe can be repurposed for censorship, and a new benchmark shows that LLMs can mimic human ethical conclusions while reasoning from entirely different moral foundations. For engineering and product leaders, the message is consistent: surface-level agreement with expected outputs is not a reliable proxy for trustworthy behavior, and the gap between "it looks right" and "it is right" is where the real risk lives.
All the best, the KYFEX team
“Agreement with human judgments is a common proxy for evaluating the alignment of large language models (LLMs). Yet agreement in final labels does not show that human annotators and models rely”
arXiv cs.AI
Quick hits
AI in the courtroom: governance gaps made visible
Amazon uses Twitch content to train AI unless streamers opt out
Twitch quietly enabled use of streamer content for AI training, triggering backlash when thousands of users discovered the default was opt-in rather than opt-out.
Why it matters: Default-on data harvesting for AI training is becoming a flashpoint: product teams should treat consent architecture as a first-class design decision, not a legal afterthought.
Alignment research: competing visions, real tradeoffs
Three papers published this week pull alignment thinking in different directions, and together they force a harder question: are we building AI that genuinely reasons well, or AI that just produces outputs we recognize?
Alignment methods may double as a censor's toolkit, researchers warn
A position paper argues that the same techniques used to prevent harmful AI outputs are dual-use technologies that authoritarian actors could repurpose to suppress legitimate speech at scale.
Why it matters: Teams deploying content moderation or output-filtering pipelines should audit whether those systems could be misconfigured or misappropriated to block protected expression, not just harmful content.
LLMs can agree with human ethical judgments while reasoning from different moral grounds
New research shows that matching human labels on ethical questions does not mean an LLM is using the same moral reasoning, meaning agreement-based alignment evaluations may be systematically misleading.
Why it matters: If your safety or values evaluation relies on label matching, today's paper is a direct challenge to that methodology: you need probes that surface reasoning paths, not just final answers.
Position paper: practical alignment must mirror human reasoning, not just outcomes
This paper argues that in high-stakes settings, AI alignment methods should be designed to replicate human reasoning processes, not merely produce outputs that look human-approved.
Why it matters: For teams building decision-support tools in regulated domains, this is a design constraint worth taking seriously now, before auditors and regulators make it mandatory.
This course will teach you about computer vision ML using libraries and models from the HF ecosystem
Hugging Face · Free
Put it to work
Try this today
Audit your AI pipeline for prompt-injection exposure
You are a security reviewer. I will describe an AI-assisted workflow that processes external text input (such as user documents, emails, or web content). Identify every point where untrusted text could reach the model as an instruction rather than as data. For each risk point, suggest one concrete mitigation. Workflow description: [paste your workflow here]
Why it helps: The court prompt-injection story is a reminder that any pipeline ingesting user-supplied documents is a potential attack surface, and most teams have not formally mapped where trust boundaries exist.
Before you ship it
The risk
LLMs that match expected ethical labels can still be reasoning from entirely different moral foundations, meaning label-agreement evaluations give a false sense of alignment safety in high-stakes deployments.
Do this
Supplement label-matching evaluations with reasoning-path probes: ask the model to explain its moral logic step by step, then review whether those explanations are consistent with the values your organization actually intends to encode.
Ready to ship AI, not just read about it?
KYFEX designs and builds production AI for teams that need it working, not just demoed. Tell us what you're working on and we'll bring the engineering.
