CV  ·  LinkedIn

Hi 👋 I'm Liming Liu, an independent researcher and software engineer based in Vancouver, Canada. I study how agents can use and compose capabilities reliably, and how evaluation can reveal where their behavior breaks down.

My interests span reliable autonomous agents, agent safety evaluation, capability learning and composition, and failure diagnosis. My current projects examine software and enterprise-workflow agents through controlled experiments, trajectory inspection, and state-based evaluation.

Delegation reliability · Experimental controls · Evaluation validity

Question: when a Planner hands a task to a Developer, what can outcome differences tell us about reliability—and does restoring the original requirement help beyond simply trying again?

• Designed a paired study on 30 SWE-Gate tasks, with 90 Planner runs and 270 Developer runs, followed by a separate 270-run prospective retry-control study.
• Audited trajectories and constraint evaluators, addressed network access as a potential confound, and separated retrospective rescoring from new model execution.
• In the follow-up, source attachment and handoff-only retry each achieved 66/90 joint successes: a 0.0 percentage-point difference, with a 95% task-bootstrap interval of −7.8 to +8.9 points. This established neither an average advantage nor equivalence.

Research lesson: a later success does not identify the cause of recovery. Controls and measurement audits matter as much as the initial observation.

Explore the study, methods and reproducible results →

Capability learning · Workflow simulation · Interface generalization

Question: can a small language-model agent compose workflow decisions it has learned individually into successful, unseen workflows?

• Built a deterministic, offline Jira–GitHub workflow simulator with state-based checks for task outcomes and operational constraints.
• Fine-tuned SmolLM2-360M Base and Instruct models across three seeds each; all six met the project's atomic-skill proficiency criteria at epoch 4.
• Three qualified Base-model checkpoints completed none of 16 shared unseen composition scenarios each, despite producing 510/510 structurally valid actions. Controlled comparisons identified sensitivity to the displayed and permitted tool interface, motivating a shared-interface proficiency requirement for follow-up evaluation.

Research lesson: valid action structure and success on isolated skills do not establish workflow competence. Interface changes must be controlled before attributing failure to composition.

Explore the simulator, experiments and research process →

Both are independent research projects. The linked repositories document their methods, evidence boundaries and AI assistance; they are not presented here as peer-reviewed publications.

My engineering background motivates questions about agents operating across tools, interfaces and changing workflow states. Through research, I am learning to turn those questions into testable comparisons and conclusions that reflect the limits of the evidence.

I'm interested in PhD opportunities and research conversations around reliable autonomous agents and agent evaluation.

CV · LinkedIn