Skip to content

7 minute read

GR00T N1.7 runs 24.6% faster on H100 with Fenon.

Fenon runs NVIDIA’s GR00T N1.7 in 16.1 ms per action chunk on an H100. NVIDIA’s optimized TensorRT path takes 21.3 ms on the same machine.

GR00T N1.7 is NVIDIA’s open foundation model for robot manipulation. Each call takes camera images, the robot’s state and a language instruction, and returns a chunk of actions. While that call runs, the robot keeps acting on the previous observation, so inference latency directly affects control.

We ran the same GR00T N1.7 checkpoint on the Fenon runtime and on NVIDIA’s optimized TensorRT path, on the same GPUs with the same inputs. On H100 we didn’t change the weights, the numerical precision or the four denoising steps.

GR00T N1.7 · Fenon runtime vs NVIDIA TensorRTMeasured
Lower median latency than NVIDIA TensorRT on H100
24.6%
Speedup on H100, p95 down from 21.64 to 16.33 ms
1.33×
Lower median latency on A100, 1.75× the calls per second
42.8%
Gripper decisions changed; worst action error +0.36%
0
Headline results. Both runtimes used the same machine, checkpoint and inputs.

Results.

We timed 180 complete calls per runtime, from raw camera frames to decoded actions, alternating between the runtimes across three processes.

Median latency · H100 and A100Measured
Median latency per action chunk.
MetricNVIDIA TensorRTFenon runtimeChange
H100 median21.34 ms16.08 ms−24.6%
H100 p9521.64 ms16.33 ms−24.5%
A100 median68.48 ms39.18 ms−42.8%
A100 p9570.55 ms41.60 ms−41.0%
A100 throughput14.6 calls/s25.5 calls/s1.75×

On both GPUs the p95 fell by about as much as the median: 24.5% on H100 and 41.0% on A100.

Inputs and output quality.

Camera frames fed to the policyDataset images
Simulated tabletop seen from the front camera: a robot arm above three mugs and two plates on a wooden table.
LIBERO · front
The same scene from the wrist camera: mugs and a plate seen from just above the table.
LIBERO · wrist
Real lab scene from an exterior camera: a robot arm beside a sink with a spray bottle and cleaning supplies.
DROID · exterior
The same real lab scene from the robot's wrist camera.
DROID · wrist
Camera frames used in the benchmarks: simulated LIBERO for H100, a real DROID lab scene for A100. LIBERO frames are stored rotated in the dataset.

Each call gets fresh images, state and noise. Nothing is cached between calls. We fixed the quality checks before timing anything: compared with NVIDIA’s output, actions had to keep a cosine similarity above 0.999, no action dimension could degrade by more than 1% against the recorded demonstrations, and no gripper open/close decision could change.

CheckLimitH100A100
Minimum cosine similarity> 0.9990.99999680.99966
Largest error change in one action dimension< 1%+0.36%+0.13%
Gripper decisions changed00Not measured

Where the time went.

We profiled NVIDIA’s path before changing anything. On A100, the GPU was idle for the first 12.2 ms of each call while the CPU prepared images, and for the last 11.2 ms while the CPU decoded actions. On H100, no kernel was running for 9.5 ms of a 22.7 ms profiled call.

A100 · one call, GPU kernel activity (Nsight)Measured
NVIDIA TensorRT · 71.4 ms · 2,245 kernels12.2 ms before the first kernel11.2 ms after the lastFenon runtime · 41.7 ms · 2,029 kernels3.4 ms before the first kernel1.4 ms after the last0 ms15 ms30 ms45 ms60 ms75 ms
Kernel activity in one profiled A100 call. Profiling adds overhead, so these calls are slower than the timed ones.

Most of our changes target that idle time:

  • Scheduling. The vision, language and action stages are queued on the GPU back to back. The CPU waits once, when the actions are ready, instead of after every stage.

  • Repeated work. Some work ran on each of the four denoising passes. It now runs once per call.

  • Input preparation. Image preparation and the language model’s input setup now run on the GPU.

  • Checks at each step. We timed each change on its own and kept it only if it passed the quality checks.

H100 · median latency, change by changeMeasured
12141618202221.06 msBaseline18.93 msAsync scheduling16.90 msStatic shapes16.04 msAction model15.64 msCPU caching
  • Fenon runtime
  • NVIDIA TensorRT
Median latency / ms (axis from 12)
Median H100 latency after each change, from the test run in which that change was accepted.

Approaches we evaluated and rejected.

We also built lower-precision and fewer-step versions of the model. All of them were faster than NVIDIA’s path, and all of them failed the quality checks, so none are in the runtime.

H100 · speed vs worst action errorMeasured
Time saved against action error. Only the Fenon runtime is under the 1% limit. Fenon runtime: 5.26 ms faster, largest error change +0.36%; One denoising step: 6.27 ms faster, largest error change +9,385%; Two denoising steps: 4.08 ms faster, largest error change +1,542%; Action FP8, all 32 blocks: 1.89 ms faster, largest error change +69.20%; Vision FP16: 1.40 ms faster, largest error change +6.06%; Vision FP8: 1.60 ms faster, largest error change +2.96%; Language FP8: 0.24 ms faster, largest error change +7.69%; Language FP16: 0.30 ms faster, largest error change +3.85%; Action FP8, blocks 4, 5, 16, 31: 0.28 ms faster, largest error change +1.88%; Action FP8, block 13: 0.01 ms faster, largest error change +3.85%.0.1%1%10%100%1k%10k%0 ms1 ms2 ms3 ms4 ms5 ms6 ms7 msFaster than the version it was built on →Largest error change →Over 1%: rejectedUnder 1%: acceptedFP8 or FP16 in the language model, or FP8 in 1–4 action blocks: under 0.3 ms fasterFenon runtime (vs NVIDIA)One denoising stepTwo denoising stepsAction FP8, all 32 blocksVision FP16Vision FP8
Each marker is one candidate change. The horizontal axis shows latency saved against the version it was built on; the vertical axis shows the largest error increase in any action dimension. Hover over or select a marker for details.
  • FP8 and FP16. Quantizing a small number of layers produced little gain. FP8 in four of the 32 action blocks saved 0.28 ms, and block 13 alone saved 0.01 ms; both still failed. FP8 in all 32 blocks saved 1.89 ms and degraded gripper error by 69%. The language model saved under 0.3 ms in FP16 or FP8.

  • Fewer denoising steps. Two steps brought latency to 13.0 ms and one step to 10.8 ms, but gripper error went up by more than 1,500% and 9,000%.

  • Cosine similarity. Several candidates scored above 0.9999 and still degraded one action dimension by more than 1%, so each dimension is checked separately.

A100.

On A100 we used the base N1.7-3B checkpoint on a DROID observation with two cameras and two frames per camera, 461 tokens in total. Median latency went from 68.48 ms to 39.18 ms. Here the runtime runs the vision encoder in FP16, which shifts the actions slightly; the largest change in any action dimension was 0.13%.

A100 · every timed call, cumulativeMeasured
0%25%50%75%100%3545556575Fenon runtimeNVIDIA TensorRT
  • Fenon runtime
  • NVIDIA TensorRT
Latency / ms
All 180 timed calls per runtime on A100. The slowest Fenon call (43.8 ms) was faster than the fastest NVIDIA call (67.5 ms).

How we measured.

Both runtimes ran on the same machine, with the same checkpoint and inputs, in alternating processes.

SetupH100A100
GPUH100 80 GB HBM3A100-SXM4 40 GB
ModelGR00T N1.7-LIBERO (libero_10)GR00T N1.7-3B
InputLIBERO, two 256×256 camerasDROID, two cameras × two frames
Vision-language tokens156461
Denoising steps44
Action horizon40 (16 decoded actions)40
BaselineNVIDIA optimized TensorRTNVIDIA optimized TensorRT
Fenon precisionUnchanged (FP32 vision, BF16 language and action)FP16 vision, BF16 language and action
SoftwarePyTorch 2.9.0, TensorRT 10.15.1.29PyTorch 2.9.0, TensorRT 10.15.1.29
Calls per runtime180 (3 processes × 60, 20 warmups each)180 (3 processes × 60, 20 warmups each)
Dates18–19 October 202618–19 October 2026

Latency is one complete, warmed, local call, from raw observation to decoded actions. Startup, network and queueing are excluded. Quality checks are offline; closed-loop task success has not been measured yet.

Next steps.

  • B300. We will run the same comparison on NVIDIA B300, including block-scaled FP8 for the action model.

  • More tasks. We will evaluate both runtimes side by side on closed-loop pick-and-place tasks and report task success alongside latency.

  • Live API. We will make GR00T N1.7 on Fenon available through a hosted API designed as a drop-in replacement for existing inference endpoints, with the same inputs and action outputs. Reported latency will include network and queueing.

  • More models. We will extend the runtime to other robot foundation models.