This repository was archived by the owner on Mar 23, 2026. It is now read-only.

tests: run store checks in Community against Pro workflow#13721

Merged

bentsku merged 10 commits into

mainfrom

comm-store-police

Feb 11, 2026

bentsku commented Feb 9, 2026 •

edited

Loading

Contributor

Motivation

We are moving towards a serialization framework based on type hints for our Models.

This PR adds a special pytest plugin in our Community against Pro Tests runs (as those are the only one with access to Pro) which will try to serialize all stores touched during the test after each test. If some data stored in that store is not serializable, the test will fail with an error message explaining why.

This is currently only the case for services which have been onboarded, such a SQS or S3. See for example the Lambda PR: #13707

It already picked up a few issues, explained in the Changes section

Changes

Plugin addition

add the pytest plugin only if the Pro code is available
- the plugin is added to only test Avro serialization, with pickle serialization disabled as it slows down the pipeline too much, leading to timeout

Service fixes:

API Gateway: fix an issue where we imported a DocumentationPart.properties, we would store it as a dictionary instead of a string, with validated tests
Events: fixed type hints of FormattedEvent.time which should have been a str
KMS: fix type hint of KMSKey.previous_keys which can hold the key material, so it is list[bytes | None] and KmsCryptoKey.key_material which can be None
S3: fix two nullable fields, which can be None in very specific conditions (manually overridden for some special behavior)
SQS: MessageMoveTask.destination_arn is nullable

Tests

Related

bentsku added this to the 4.14 milestone

bentsku self-assigned this

bentsku added semver: patch docs: skip notes: skip labels

github-actions Bot commented Feb 9, 2026 •

edited

Loading

Test Results - Preflight, Unit

23 140 tests ±0 21 276 ✅ ±0 6m 30s ⏱️ -2s
1 suites ±0 1 864 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit a826d0e. ± Comparison against base commit 8cda50b.

♻️ This comment has been updated with latest results.

github-actions Bot commented Feb 9, 2026 •

edited

Loading

S3 Image Test Results (AMD64 / ARM64)

2 files 2 suites 8m 25s ⏱️
559 tests 507 ✅ 52 💤 0 ❌
1 118 runs 1 014 ✅ 104 💤 0 ❌

Results for commit a826d0e.

♻️ This comment has been updated with latest results.

github-actions Bot commented Feb 9, 2026 •

edited

Loading

Test Results (amd64) - Integration, Bootstrap

5 files 5 suites 2h 35m 50s ⏱️
5 227 tests 4 882 ✅ 345 💤 0 ❌
5 233 runs 4 882 ✅ 351 💤 0 ❌

Results for commit a826d0e.

♻️ This comment has been updated with latest results.

bentsku force-pushed the comm-store-police branch from 2fb2148 to 5bff7d4 Compare

February 10, 2026 00:55

github-actions Bot commented Feb 10, 2026 •

edited

Loading

Test Results (amd64) - Acceptance

7 tests ±0 5 ✅ ±0 3m 3s ⏱️ +4s
1 suites ±0 2 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit a826d0e. ± Comparison against base commit 8cda50b.

♻️ This comment has been updated with latest results.

github-actions Bot commented Feb 10, 2026 •

edited

Loading

Test Results - Alternative Providers

407 tests 274 ✅ 3m 41s ⏱️
1 suites 133 💤
1 files 0 ❌

Results for commit a826d0e.

♻️ This comment has been updated with latest results.

github-actions Bot commented Feb 10, 2026 •

edited

Loading

LocalStack Community integration with Pro

2 files ±0 2 suites ±0 2h 2m 42s ⏱️ + 4m 53s
5 203 tests ±0 4 851 ✅ ±0 351 💤 ±0 1 ❌ ±0
5 205 runs ±0 4 851 ✅ ±0 353 💤 ±0 1 ❌ ±0

For more details on these failures, see this check.

Results for commit a826d0e. ± Comparison against base commit 8cda50b.

♻️ This comment has been updated with latest results.

bentsku force-pushed the comm-store-police branch 2 times, most recently from d31f763 to 6cbc5ad Compare

February 10, 2026 19:50

bentsku marked this pull request as ready for review

February 10, 2026 22:35

bentsku requested review from aidehn, baermat, cloutierMat, gregfurman, k-a-il, maxhoheiser and thrau as code owners

February 10, 2026 22:35

bentsku assigned giograno

bentsku requested a review from giograno

February 10, 2026 22:35

aidehn reviewed

View reviewed changes

tests/aws/conftest.py

+                      if not test_config.TEST_SKIP_LOCALSTACK_START and not is_aws_cloud():
+                          # this directly accesses LocalStack state in memory, so it is not worth running in tests against external
+                          # instances
+                          pluginmanager.register(StoreSerializationCheckerPlugin(with_pickle=False))

aidehn Feb 11, 2026

Contributor

OOC why do we need with_pickle=False here?

bentsku Feb 11, 2026

Contributor Author

Ah my bad forgot to update the PR description: I had to update the plugin implementation to not test pickling serialization because it made the pipeline timeout due to its cost 😅 and we have done without it for years now only added the new serialization framework checks 👍

aidehn approved these changes

View reviewed changes

aidehn left a comment

Contributor

Looks good!

cloutierMat reviewed

View reviewed changes

cloutierMat left a comment

Contributor

Thanks for improving apigateway while you were there! 🙏

I do have a couple questions regarding the test. It seems that maybe an earlier snapshot skip remained and might not be relevant anymore?

tests/aws/services/apigateway/test_apigateway_api.py Outdated

tests/aws/services/apigateway/test_apigateway_api.py Outdated

bentsku added 3 commits

February 11, 2026 09:59


          trying things out

edda73a


          fix S3 model

5b033c4


          fix Events model

11eaf3e

bentsku added 6 commits

February 11, 2026 09:59


          fix APIGW DocumentationParts typing

f1fde26


          fix KMS model typing

e194ae9


          disable pickle police


          fix SQS model

cc03052


          fix S3 ChecksumAlgo enum for SSE-C Multipart

c47ff4d


          address APIGW test comments

6dace55

bentsku force-pushed the comm-store-police branch from 6cbc5ad to 6dace55 Compare

February 11, 2026 09:00


          fix snapshot skip for APIGW

a826d0e

giograno approved these changes

View reviewed changes

giograno left a comment

Member

Great work! Having this in community is going to be super important for us 💯

bentsku commented Feb 11, 2026

Contributor Author

Test failure is unrelated, addressed by #13743

bentsku merged commit 931e0a2 into main

47 of 49 checks passed

bentsku deleted the comm-store-police branch

February 11, 2026 11:21

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

baermat Awaiting requested review from baermat

gregfurman Awaiting requested review from gregfurman

thrau Awaiting requested review from thrau thrau is a code owner

k-a-il Awaiting requested review from k-a-il k-a-il is a code owner

maxhoheiser Awaiting requested review from maxhoheiser

3 more reviewers

cloutierMat cloutierMat left review comments

giograno giograno approved these changes

aidehn aidehn approved these changes

Labels

docs: skip notes: skip semver: patch