Evaluating Unity ECS cohesion by separating measurement, observation, and decision

Experiment card

The important part of this experiment is that the adoption decision is not explained by a single number. The direction of the CPU Main difference changed with entity count, and the per-run ranges overlapped. A fixed-condition visual check, meanwhile, made local grouping look slightly stronger. The value was therefore adopted because it added the intended visual behavior without revealing a visible conflict with existing behavior—not because performance improved.

Conclusions

• With 1,000 entities, the average CPU Main difference with cohesion enabled was +0.026 ms; with 5,000 entities, it was -0.086 ms.
• The direction was inconsistent and the ranges from three runs per condition overlapped, so the experiment does not establish an increase or decrease in cost.
• Average Unity reserved memory did not differ between disabled and enabled cohesion at either entity count.
• In a fixed-seed visual comparison of 1,000 entities moving to the same target, local grouping looked slightly stronger with cohesion enabled.
• Through approximately six seconds after arrival, no cohesion-specific overdensity, expansion and contraction of the group, or oscillation around the center was observed.
• Acceleration 1.0 is an initial value only for this minimal Boids-lite setup. It is neither a general recommendation nor a value justified by a performance difference.

Separate measurement, observation, and decision

An experiment report should not merge these three types of result.

“The appearance improved,” “the average changed,” and “the value became the default” are separate claims. The core lesson is not to turn a small average difference into a performance improvement, a short visual check into proof of stability, or one project's initial value into a general recommendation.

A/B measurement conditions

The same Windows Standalone x64 Development Build was used, with only cohesion toggled.

Before measurement, batchmode compilation, 106 Edit Mode tests, the Windows Development Build, and successful completion of every visible Player run were checked. The measurements are whole-Player frame metrics, not a Profiler Marker isolated to the cohesion computation.

Measured results

The following values are averages from three runs per condition.

The enabled average increased at 1,000 entities and decreased at 5,000. The recorded per-run ranges also overlapped in both cases. Run-to-run variation may therefore be as large as or larger than the incremental cohesion cost in this test. These measurements do not support either “cohesion made it faster” or “cohesion made it slower.”

GPU FrameTiming produced unrealistic outliers in multiple runs and was excluded from the conclusion. Rejecting a metric that cannot be trusted, instead of reporting every available number, is part of the result.

Visual observation

The visual comparison moved 1,000 entities generated from fixed seed 20260816 toward the same screen-space target. Entity count was preserved, separation and alignment remained enabled, and cohesion acceleration was 1.0. Motion was checked around 0.5 seconds after setting the target and the settled state around six seconds after arrival.

The observations were limited to the following:

• Local grouping looked slightly stronger with cohesion enabled.
• Both conditions formed a dense group around the single target after arrival.
• Through approximately six seconds, no enabled-only overdensity, expansion and contraction of the group, or oscillation around the center was observed.
• No visible problem was found in target arrival, minimal avoidance from separation, or existing enemy behavior.

These are qualitative observations, not measurements of local density, flock shape, or nearest-neighbor distance. No publishable side-by-side image or video was retained, so the visual result is not presented as independently inspectable evidence. “Not observed” means only that the problem was not found during the documented short window and conditions.

Adoption decision

This minimal Boids-lite setup adopted cohesion acceleration 1.0 as its initial value. It is weaker than alignment at 2.0 and separation at 6.0, allowing the movement target to remain the primary direction while drawing allies slightly toward a local center.

The reason for adoption was that the fixed-condition check showed the intended visual change without revealing a visible conflict with existing behavior. The CPU Main average differences were not used to justify the decision. Different densities, neighbor counts, game rules, and rendering loads may require a different value.

Turning a small difference into the next experiment

When the direction of a difference is inconsistent and run ranges overlap, it is safer not to decide feature adoption or optimization from averages alone. A follow-up can add evidence according to the question being asked.

• Increase the run count and alternate or randomize execution order.
• Inspect percentiles and short Profiler captures in addition to averages.
• Add a cohesion-specific Profiler Marker to separate its cost from whole-Player variation.
• Record the same local-density, nearest-neighbor, overlap-count, and arrival-time metrics in both conditions.
• Retain publishable images or video at matching timestamps so visual observations can be checked again.

The goal is not to force a stronger conclusion out of a small difference. The measurement can remain “cost change not established,” the observation can remain “the appearance changed slightly under limited conditions,” and the decision can remain “an initial value was selected for the current prototype.”

Caveats

• The result is limited to one Windows Development Player, the documented entity counts, and three runs per condition.
• CPU Main is a whole-Player average, not the execution time of cohesion alone.
• Equal reserved memory does not prove that temporary allocators or in-frame allocations were identical.
• The visual comparison was not quantitative and has no publishable image or video.
• Finding no problem through approximately six seconds does not establish long-duration stability or behavior with other target layouts.
• Acceleration 1.0 is a decision for this setup, not a recommendation for Unity ECS or Boids in general.

Related resources

• Measuring 1,000 to 5,000 Unity ECS entities reproducibly in a Windows Player (Japanese)
• A/B testing same-faction separation with a reused spatial index in Unity ECS (Japanese)
• Profiling your application in Unity