Skip to content

Feat: add engine throughput and policy evaluation suites#70

Merged
rian-be merged 4 commits into
mainfrom
develop
Jul 2, 2026
Merged

Feat: add engine throughput and policy evaluation suites#70
rian-be merged 4 commits into
mainfrom
develop

Conversation

@rian-be

@rian-be rian-be commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR expands benchmark coverage for ModularityKit.Mutator by adding dedicated engine throughput and policy evaluation suites, then reorganizes the benchmark project into scenario-focused folders.

Highlights

Core engine throughput benchmarks

  • Add dedicated engine benchmarks under Benchmarks/Engine.
  • Measure single mutation throughput separately from batch throughput.
  • Compare throughput across explicit StateSize and BatchSize parameters.
  • Split commit, mode, batch, and throughput scenarios into focused benchmark classes.

Policy evaluation benchmarks

  • Add dedicated policy benchmarks under Benchmarks/Policy.
  • Measure no policy baseline, single synchronous policy, single asynchronous policy, and mixed multi policy evaluation paths.
  • Keep helper mutations and policies local to the benchmark project.

Benchmark project structure

  • Replace the previous all-in-one engine benchmark class with smaller concern-specific classes.
  • Add support helpers scoped to Engine and Policy folders.
  • Update namespaces and XML docs to match the physical benchmark layout.
  • Refresh Benchmarks/README.md with the new benchmark filters and suite descriptions.

Why

Issue #56 asks for throughput focused visibility into the core mutation runtime across single and batch execution paths, including meaningful state size variation.

Issue #57 asks for explicit policy overhead benchmarks across sync, async, and multi-policy execution modes.

This PR addresses both issues while cleaning up the benchmark project structure so the suite is easier to extend and compare over time.

Checks

  • dotnet build Benchmarks/ModularityKit.Mutator.Benchmarks.csproj -c Release
  • sample local BenchmarkDotNet runs for engine throughput and policy evaluation scenarios

Closes

@github-actions github-actions Bot added documentation Documentation updates and additions ci CI/CD and repository automation changes benchmark Benchmark coverage and performance measurement changes performance Performance improvements or regressions labels Jul 2, 2026
@rian-be rian-be changed the title benchmarks: add engine throughput and policy evaluation suites Feat: add engine throughput and policy evaluation suites Jul 2, 2026
@rian-be rian-be merged commit ade5baf into main Jul 2, 2026
28 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

benchmark Benchmark coverage and performance measurement changes ci CI/CD and repository automation changes documentation Documentation updates and additions performance Performance improvements or regressions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Benchmark]: Measure policy evaluation overhead [Benchmark]: Measure mutation engine throughput

1 participant