Tests: add helper Stress attribute for stress testing#18452
Conversation
✅ No release notes required |
|
Good stuff. I think VS allows for "run until failure" but not sure if it stressing the thread pool this much, and also it's not x-plat. Would you mind adding a line about that to the testing guide? :) |
…ize.fs Co-authored-by: Petr <psfinaki@users.noreply.github.com>
The difference is VS "run until failure" will execute sequentially if we select only one test case. For example it couldn't reproduce the failure I linked above, while with Hmm, now I think the naming could be better. |
|
I'm not sure if this should to TESTGUIDE or DEVGUIDE? There was a section already to which I added. |
psfinaki
left a comment
psfinaki
left a comment
There was a problem hiding this comment.
I think this is good enough for now :) Thanks!
|
Speaking of naming, I think "Stress" would be even better than "Repeat". |
majocha
changed the title
Repeat attribute for stress testingStress attribute for stress testing
github-project-automation
Bot
moved this from New
to In Progress
in F# Compiler and Tooling
|
Makes sense! But now I'm in decision paralysis where to put such test 😄. This will be a test of test utilities, so it doesn't belong anywhere... |
Well, we can always start a testutilities test assembly. With this as the first test, hereafter, we will have no excuse for not testing the hacks we add to the test framework. |
I'm guessing many of the smoketests are only smoketests because they are testing that the test framework does something interesting. |
|
Would it be ok to just unlock existing FSharp.Test.Utilities project as a test container? |
I would do it like that as well, keep the tests close to those utilities. |
|
Tests added :) |
github-project-automation
Bot
moved this from In Progress
to Done
in F# Compiler and Tooling
4 participants
This is useful for locally stress testing cases that are flaky in parallel execution.
Example usage: replace
[<Fact>]with[<Theory; Stress(Count = 1000)>]Unlike VS "Run until failure" feature which runs sequentially, this will start the test simultaneously many times and run in parallel, stressing the threadpool.
It can be used to locally track down bugs in flaky tests.