Beyond final artifacts
Final outputs hide the process through which humans and models jointly shape task goals, requirements, and decisions.
Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration
KAIST · Carnegie Mellon University · Seoul National University
CoTrace traces how humans and LLMs jointly shape goals, requirements, and decisions across multi-turn collaboration—not only who wrote the final artifact.
Why goal attribution matters
Existing attribution methods focus on final artifacts, but in human–LLM collaboration, contribution often happens earlier: when goals are formed, refined, and made concrete through interaction.
Goals evolve across turns
As humans and models interact, goals evolve: LLMs may directly introduce new requirements, or indirectly elicit user goals by providing artifacts, options, questions, or constraints.
Users miscalibrate contribution
Without goal-level traces, users may underestimate how much AI shaped their work and over-attribute decisions to themselves.
The CoTrace pipeline
CoTrace is built around two core design choices: representing goals as checkable requirements, and tracing both direct and indirect influence across dialogue turns.
Goal and Requirement
CoTrace treats a goal as an explicit, actionable target with a desired outcome, then decomposes underspecified goals into requirements—the smallest independently checkable success predicates.
Direct and Indirect Influence
CoTrace models goal shaping as cumulative: actions can directly introduce or modify requirements, or indirectly provide context that later motivates another requirement.
Outcome & action extraction
Segment dialogue and identify outcomes, atomic actions, and whether each action shapes, executes, or plays another role.
Requirement extraction
Convert goals into independently checkable requirements and track create, revise, and delete operations over time.
Influence labeling
Link actions to requirements through direct or implicit influence, capturing both explicit decisions and subtle steering.
Contribution scoring
Aggregate influence across requirements and goal levels to summarize each participant’s role in shaping the collaboration.
Measuring · Inducing · Exposing Goal-Level Contributions
CoTrace provides three complementary uses: measuring collaborative goal shaping, inducing goal-shaping behavior at inference time, and exposing these dynamics to users.
Measuring collaborative goal shaping in the wild.
CoTrace applies goal-level attribution to real-world human–LLM collaboration logs across task types and goal specificity levels.
Setting
638 ShareChat logs across 4 task categories.
Measure
Goal-level contribution across parent goals, child goals, requirements, and direct / indirect influence links.
Findings
Inducing and evaluating goal shaping at inference time.
CoTrace asks whether model goal-shaping can be amplified through intervention, and whether amplified goal-shaping improves collaboration outcomes.
Setting
Controlled CoGym simulations across related-work writing, travel planning, and tabular analysis tasks.
Findings
Improving user awareness and intentionality.
CoTrace-viewer makes contribution dynamics legible, helping users notice aspects of collaboration they had not previously been aware of.
P6
“I didn’t necessarily make the micro decisions.”
P3
“Learning about the indirect influence really surprised me.”
P1
“I was not doing this cognitive reflection yet.”
P1
“It was surprising to me how much the tool was making decisions without me explicitly stating them.”
P2
“I feel like I should be way more specific in my prompting.”
Findings
Citation
@misc{kim2026cotrace,
title = {"I didn't Make the Micro Decisions": Measuring, Inducing, and Exposing Goal-Level AI Contributions in Collaboration},
author = {Kim, Eunsu and Mindel, Jessica R. and Kim, Kyungjin and Wu, Sherry Tongshuang},
year = {2026},
eprint = {2605.21363},
archivePrefix= {arXiv},
primaryClass = {cs.CL},
doi = {10.48550/arXiv.2605.21363}
}