work / engineering and ai
Final-year project
Joint research on Bayesian scoring and conformal risk control for a small LLM, evaluated on a held-out synthetic tool-use testbed under Prof. XU Hong.
- ≈80%
- fewer tasks with rule violations vs a prompted 4B LLM, on a held-out synthetic tool-use testbed
The problem
Small LLM agents that run locally are cheap and private, but they break the rules they are given. Prompting harder only goes so far.
What I built
Final-year project (Prof. XU Hong): co-developed Bayesian scoring and conformal risk control for a 4B LLM, evaluated on a held-out synthetic tool-use testbed.
Result
On that held-out synthetic testbed, tasks with rule violations fell by about 80% versus the prompted-agent baseline.
How it works
The layer sits between the agent and its actions. It combines Bayesian scoring with conformal risk controlconformal risk controlA way to set a decision threshold from held-out data so that an error rate stays below a chosen target. to decide whether an action passes the configured decision rule.
Limits
- The joint research is unpublished, so this page is an overview only, with no code link. The measured reduction is specific to the synthetic testbed; it is not a deployment benefit or general safety guarantee.