44/50pick up the black bowl between the plate and the ramekin and
place it on the plate47/50pick up the black bowl next to the ramekin and place it on the
plate50/50pick up the black bowl from table center and place it on the
plate48/50pick up the black bowl on the cookie box and place it on the
plate50/50pick up the black bowl in the top drawer of the wooden cabinet
and place it on the plate16/50pick up the black bowl on the ramekin and place it on the plate
49/50pick up the black bowl next to the cookie box and place it on
the plate49/50pick up the black bowl on the stove and place it on the plate
42/50pick up the black bowl next to the plate and place it on the
plate45/50pick up the black bowl on the wooden cabinet and place it on
the plate
50/50pick up the alphabet soup and place it in the basket
50/50pick up the cream cheese and place it in the basket
50/50pick up the salad dressing and place it in the basket
50/50pick up the bbq sauce and place it in the basket50/50pick up the ketchup and place it in the basket49/50pick up the tomato sauce and place it in the basket
50/50pick up the butter and place it in the basket50/50pick up the milk and place it in the basket50/50pick up the chocolate pudding and place it in the basket
50/50pick up the orange juice and place it in the basket
50/50open the middle drawer of the cabinet48/50put the bowl on the stove47/50put the wine bottle on top of the cabinet48/50open the top drawer and put the bowl inside50/50put the bowl on top of the cabinet50/50push the plate to the front of the stove49/50put the cream cheese in the bowl50/50turn on the stove49/50put the bowl on the plate47/50put the wine bottle on the rack
34/50put both the alphabet soup and the tomato sauce in the basket
41/50put both the cream cheese box and the butter in the basket
48/50turn on the stove and put the moka pot on it50/50put the black bowl in the bottom drawer of the cabinet and
close it44/50put the white mug on the left plate and put the yellow and
white mug on the right plate49/50pick up the book and place it in the back compartment of the
caddy28/50put the white mug on the plate and put the chocolate pudding to
the right of the plate39/50put both the alphabet soup and the cream cheese box in the
basket35/50put both moka pots on the stove39/50put the yellow and white mug in the microwave and close it
Training-free k-NN rollouts on all 40 LIBERO tasks (2× speed).
Abstract
Vision-language-action (VLA) models now exceed 95 % success on the LIBERO simulation benchmark, yet it
remains unclear how much of this performance requires generalization beyond the training demonstrations. In
this work, we show that retrieving nearby demonstrations and reusing their action chunks achieves a success
rate of 91.7 % on LIBERO without policy training. Under LIBERO-Plus perturbations, retrieval shows the
smallest drop of all evaluated policies. Across nine publicly available VLAs, fine-tuning mostly makes the
vision-language backbone a better retrieval key, and the actions of the VLAs remain close to the
demonstrations. In addition, we introduce Local Action Consistency (LAC), an offline metric that measures
whether nearby observations in representation space are also associated with similar future actions. We
show that LAC tracks the success rate of retrieval and can guide the training of a retrieval key.
Method
Overview of our evaluation pipeline. We use the same rollouts for all evaluations.
(a) k-NN: retrieves the nearest demonstration frames and combines their corresponding action
chunks.
(b) VLA: predicts an action chunk directly through its backbone and action head.
Local Action Consistency (LAC)
Do similar representations correspond to similar future actions? We introduce an offline metric that tests
this without robot rollouts. For each query, it measures whether nearby demonstrations in representation space
have more similar future action chunks than other demonstrations of the same task.
LAC compares the average future-action distance from a query to its k nearest neighbors in
representation space with the average distance to demonstrations from the same task.
Results
1Retrieval reaches 91.7 % success on LIBERO
and remains robust to perturbations
Without a learned policy, the k-NN is on par with π0-FAST on LIBERO. On
LIBERO-Plus, the multimodal k-NN outperforms 7 of the 9 evaluated VLAs and
shows the smallest drop in success rate under perturbations.
k-NN (ResNet) 88.7 %
training-free
Frozen ResNet-18 features. Retrieval within the same task.
k-NN (Multimodal) 91.7 %
training-free
+ robot state, language, time, and history. Retrieval across all tasks.
k-NN (LAC key) 93.0 %
learned key
Multimodal retrieval with a LAC-trained projection of ResNet features.
LIBEROLIBERO-Plusk-NN (ours)VLA
Success rate (%)
20406080100
Results on the 2,000 eval episodes of LIBERO and the same 420 variants (60 per
perturbation) of LIBERO-Plus for every policy.
2Retrieval-based policies can outperform fine-tuned
action heads on VLAs
We replace each VLA's action head with a k-NN keyed on its own frozen VLM. For some VLAs, this
k-NN outperforms the action head (2 of 9 on LIBERO, 4 of 9 on LIBERO-Plus), which suggests the VLM
already encodes enough information to generate the actions.
k-NN before fine-tuningk-NN after fine-tuningVLA
Success rate (%)vs VLA
0255075100
k-NN success rate using the fine-tuned VLM backbone vs the complete VLA (%pt).
GR00T's VLM is frozen, hence before = after.
3Fine-tuning on LIBERO reshapes VLM representations and often improves
retrieval
Better retrieval under perturbations for 7 of 8 fine-tuned VLAs.
Closer in task progress: nearest neighbors from other demonstrations are closer in
episode progress for most models.
Less scene-dependent: representations depend less on the LIBERO suite.
One exception, π0:
representations collapse into task clusters, and
k-NN success drops from 86.9 % to 17.9 %.
t-SNE of the same 4,150 demonstration frames, colored by episode progress.
Top: before fine-tuning.
Bottom: after fine-tuning (LAC training for ResNet-18).
Values below each map show the median progress gap to the nearest same-task frame from another
demonstration.
4
VLA-generated actions remain close to simple k-NN retrieval
Although VLAs generate actions using learned policy heads, their predictions often closely resemble those of
a simple k-NN retrieval.
For 5 of 8 VLAs, k-NN predictions are more similar to VLA-generated actions than to held-out
ground-truth actions.
Moreover, all eight VLAs produce action chunks closer to the training demonstrations than held-out
ground-truth chunks are.
5
LAC provides an offline measure of retrieval quality
Can we evaluate a retrieval key without running rollouts?
We compute LAC on held-out demonstrations and compare it with the success of k-NN policies using
the same representations.
Higher LAC generally corresponds to better retrieval performance, although it cannot reliably distinguish
keys of similar quality.
Each point represents a retrieval key: offline LAC versus LIBERO rollout success.
The dashed trend excludes fine-tuned VLM backbones, which were trained on the held-out demonstrations.
BibTeX
@inproceedings{grachev2026knn,
title={{How Much of Your VLA Is kNN}},
author={Denis Grachev and Damien LaRocque and Omar Moured},
booktitle={CoRL 2026 Workshop on G2A: From Generative Models to Robot Actions},
year={2026},
url={https://openreview.net/forum?id=OoI6kXUw6b}
}
