[ty] Track literal iterable emptiness for reachability#25222
Conversation
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 94.47%. The percentage of expected errors that received a diagnostic held steady at 89.19%. The number of fully passing files held steady at 95/134. |
Memory usage reportSummary
Significant changesClick to expand detailed breakdownprefect
trio
flake8
sphinx
|
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
possibly-unresolved-reference |
0 | 124 | 0 |
unresolved-attribute |
0 | 3 | 0 |
not-iterable |
0 | 2 | 0 |
invalid-assignment |
0 | 1 | 0 |
possibly-missing-import |
0 | 1 | 0 |
| Total | 0 | 131 | 0 |
Flaky changes detected. This PR summary excludes flaky changes; see the HTML report for details.
Raw diff (131 changes)
black (https://github.com/psf/black)
- src/black/linegen.py:943:12 warning[possibly-unresolved-reference] Name `matching_bracket` used when possibly not defined
- src/black/linegen.py:943:36 warning[possibly-unresolved-reference] Name `tail_leaves` used when possibly not defined
- src/black/linegen.py:947:9 warning[possibly-unresolved-reference] Name `head_leaves` used when possibly not defined
- src/black/linegen.py:947:28 warning[possibly-unresolved-reference] Name `matching_bracket` used when possibly not defined
- src/black/linegen.py:950:9 warning[possibly-unresolved-reference] Name `body_leaves` used when possibly not defined
- src/black/linegen.py:950:28 warning[possibly-unresolved-reference] Name `matching_bracket` used when possibly not defined
- src/black/linegen.py:953:9 warning[possibly-unresolved-reference] Name `tail_leaves` used when possibly not defined
- src/black/linegen.py:953:28 warning[possibly-unresolved-reference] Name `matching_bracket` used when possibly not defined
- src/black/files.py:104:12 warning[possibly-unresolved-reference] Name `directory` used when possibly not defined
cloud-init (https://github.com/canonical/cloud-init)
- tests/unittests/sources/test_init.py:737:26 warning[possibly-unresolved-reference] Name `filename` used when possibly not defined
colour (https://github.com/colour-science/colour)
- colour/io/tm2714.py:1698:44 warning[possibly-unresolved-reference] Name `mapping` used when possibly not defined
- colour/io/tm2714.py:1700:38 warning[possibly-unresolved-reference] Name `mapping` used when possibly not defined
- colour/io/tm2714.py:1702:21 warning[possibly-unresolved-reference] Name `mapping` used when possibly not defined
- colour/io/tm2714.py:1702:45 warning[possibly-unresolved-reference] Name `mapping` used when possibly not defined
core (https://github.com/home-assistant/core)
- homeassistant/components/opower/coordinator.py:299:24 warning[possibly-unresolved-reference] Name `stats` used when possibly not defined
- homeassistant/components/opower/coordinator.py:306:42 warning[possibly-unresolved-reference] Name `stats` used when possibly not defined
- homeassistant/components/opower/coordinator.py:308:21 warning[possibly-unresolved-reference] Name `stats` used when possibly not defined
- homeassistant/components/opower/coordinator.py:310:49 warning[possibly-unresolved-reference] Name `stats` used when possibly not defined
- homeassistant/components/opower/coordinator.py:311:44 warning[possibly-unresolved-reference] Name `stats` used when possibly not defined
- homeassistant/components/opower/coordinator.py:312:35 warning[possibly-unresolved-reference] Name `stats` used when possibly not defined
cryptography (https://github.com/pyca/cryptography)
- tests/hazmat/primitives/test_ssh.py:225:30 warning[possibly-unresolved-reference] Name `private_key` used when possibly not defined
- tests/hazmat/primitives/test_ssh.py:242:26 warning[possibly-unresolved-reference] Name `private_key` used when possibly not defined
ibis (https://github.com/ibis-project/ibis)
- ibis/backends/sql/datatypes.py:80:5 warning[possibly-unresolved-reference] Name `_attr` used when possibly not defined
- ibis/backends/sql/datatypes.py:80:12 warning[possibly-unresolved-reference] Name `_ibis_type` used when possibly not defined
- ibis/backends/sql/datatypes.py:80:24 warning[possibly-unresolved-reference] Name `_sg_type` used when possibly not defined
jax (https://github.com/google/jax)
- jax/_src/internal_test_util/test_harnesses.py:1840:11 warning[possibly-unresolved-reference] Name `shape` used when possibly not defined
- jax/_src/internal_test_util/test_harnesses.py:1841:11 warning[possibly-unresolved-reference] Name `shape` used when possibly not defined
- jax/_src/internal_test_util/test_harnesses.py:1841:31 warning[possibly-unresolved-reference] Name `shape` used when possibly not defined
- jax/_src/internal_test_util/test_harnesses.py:1845:19 warning[possibly-unresolved-reference] Name `shape` used when possibly not defined
mypy (https://github.com/python/mypy)
- mypyc/irbuild/format_str_tokenizer.py:28:55 warning[possibly-missing-import] Member `int_to_str_op` of module `mypyc.primitives.int_ops` may be missing
pandas (https://github.com/pandas-dev/pandas)
- pandas/tests/frame/indexing/test_take.py:48:13 warning[possibly-unresolved-reference] Name `df` used when possibly not defined
- pandas/tests/frame/indexing/test_take.py:50:13 warning[possibly-unresolved-reference] Name `df` used when possibly not defined
- pandas/tests/frame/indexing/test_take.py:52:13 warning[possibly-unresolved-reference] Name `df` used when possibly not defined
- pandas/tests/frame/indexing/test_take.py:54:13 warning[possibly-unresolved-reference] Name `df` used when possibly not defined
- pandas/tests/indexes/timedeltas/test_scalar_compat.py:134:35 warning[possibly-unresolved-reference] Name `r1` used when possibly not defined
- pandas/tests/indexes/timedeltas/test_scalar_compat.py:134:39 warning[possibly-unresolved-reference] Name `s1` used when possibly not defined
- pandas/tests/indexes/timedeltas/test_scalar_compat.py:135:35 warning[possibly-unresolved-reference] Name `r2` used when possibly not defined
- pandas/tests/indexes/timedeltas/test_scalar_compat.py:135:39 warning[possibly-unresolved-reference] Name `s2` used when possibly not defined
pandera (https://github.com/pandera-dev/pandera)
- tests/io/test_pandas_io.py:1545:35 warning[possibly-unresolved-reference] Name `local_dict` used when possibly not defined
- tests/io/test_pandas_io.py:1546:18 warning[possibly-unresolved-reference] Name `local_dict` used when possibly not defined
poetry (https://github.com/python-poetry/poetry)
- src/poetry/repositories/link_sources/html.py:63:77 warning[possibly-unresolved-reference] Name `metadata` used when possibly not defined
psycopg (https://github.com/psycopg/psycopg)
- tools/update_error_prefixes.py:75:22 warning[possibly-unresolved-reference] Name `default_pgroot` used when possibly not defined
pycryptodome (https://github.com/Legrandin/pycryptodome)
- lib/Crypto/SelfTest/Cipher/test_CFB.py:275:44 warning[possibly-unresolved-reference] Name `file_name` used when possibly not defined
- lib/Crypto/SelfTest/Cipher/test_CFB.py:275:55 warning[possibly-unresolved-reference] Name `new_func` used when possibly not defined
pydantic (https://github.com/pydantic/pydantic)
- pydantic/v1/networks.py:350:12 warning[possibly-unresolved-reference] Name `host` used when possibly not defined
- pydantic/v1/networks.py:355:48 warning[possibly-unresolved-reference] Name `host` used when possibly not defined
- pydantic/v1/networks.py:357:50 warning[possibly-unresolved-reference] Name `host` used when possibly not defined
- pydantic/v1/networks.py:364:50 warning[possibly-unresolved-reference] Name `host` used when possibly not defined
- pydantic/v1/networks.py:377:24 warning[possibly-unresolved-reference] Name `host` used when possibly not defined
pywin32 (https://github.com/mhammond/pywin32)
- pythonwin/pywin/idle/PyParse.py:124:5 warning[possibly-unresolved-reference] Name `ch` used when possibly not defined
rotki (https://github.com/rotki/rotki)
- rotkehlchen/tests/api/test_bitcoin_transactions.py:102:12 warning[possibly-unresolved-reference] Name `events` used when possibly not defined
- rotkehlchen/tests/api/test_bitcoin_transactions.py:131:12 warning[possibly-unresolved-reference] Name `events` used when possibly not defined
- rotkehlchen/tests/api/test_bitcoin_transactions.py:149:18 warning[possibly-unresolved-reference] Name `json` used when possibly not defined
- rotkehlchen/tests/api/test_bitcoin_transactions.py:197:12 warning[possibly-unresolved-reference] Name `events` used when possibly not defined
- rotkehlchen/tests/api/test_bitcoin_transactions.py:256:12 warning[possibly-unresolved-reference] Name `events` used when possibly not defined
- rotkehlchen/tests/api/test_solana_transactions.py:288:15 warning[possibly-unresolved-reference] Name `user_address` used when possibly not defined
- rotkehlchen/tests/api/test_solana_transactions.py:298:24 warning[possibly-unresolved-reference] Name `user_address` used when possibly not defined
- rotkehlchen/tests/integration/test_zksynclite.py:84:25 warning[possibly-unresolved-reference] Name `idx` used when possibly not defined
- rotkehlchen/tests/integration/test_zksynclite.py:101:25 warning[possibly-unresolved-reference] Name `idx` used when possibly not defined
- rotkehlchen/tests/unit/test_protocol_balances.py:541:20 warning[possibly-unresolved-reference] Name `tx_decoder` used when possibly not defined
- rotkehlchen/tests/unit/test_protocol_balances.py:990:20 warning[possibly-unresolved-reference] Name `tx_decoder` used when possibly not defined
scikit-learn (https://github.com/scikit-learn/scikit-learn)
- sklearn/covariance/tests/test_graphical_lasso.py:66:40 warning[possibly-unresolved-reference] Name `covs` used when possibly not defined
- sklearn/datasets/_twenty_newsgroups.py:341:9 warning[possibly-unresolved-reference] Name `data` used when possibly not defined
- sklearn/datasets/_twenty_newsgroups.py:342:9 warning[possibly-unresolved-reference] Name `data` used when possibly not defined
- sklearn/datasets/_twenty_newsgroups.py:343:9 warning[possibly-unresolved-reference] Name `data` used when possibly not defined
- sklearn/decomposition/tests/test_factor_analysis.py:75:9 warning[possibly-unresolved-reference] Name `fa` used when possibly not defined
- sklearn/decomposition/tests/test_factor_analysis.py:76:9 warning[possibly-unresolved-reference] Name `fa` used when possibly not defined
- sklearn/decomposition/tests/test_factor_analysis.py:77:15 warning[possibly-unresolved-reference] Name `fa` used when possibly not defined
- sklearn/decomposition/tests/test_factor_analysis.py:78:21 warning[possibly-unresolved-reference] Name `fa` used when possibly not defined
- sklearn/ensemble/tests/test_bagging.py:632:47 error[not-iterable] Object of type `None | BaggingClassifier` may not be iterable
- sklearn/ensemble/tests/test_forest.py:1326:47 error[not-iterable] Object of type `None | Any` may not be iterable
- sklearn/ensemble/tests/test_forest.py:1331:9 error[unresolved-attribute] Attribute `apply` is not defined on `None` in union `None | Any`
- sklearn/feature_extraction/tests/test_text.py:564:31 warning[possibly-unresolved-reference] Name `counts_test` used when possibly not defined
- sklearn/svm/tests/test_svm.py:80:24 warning[possibly-unresolved-reference] Name `clf` used when possibly not defined
- sklearn/svm/tests/test_svm.py:80:46 warning[possibly-unresolved-reference] Name `clf` used when possibly not defined
- sklearn/tests/test_discriminant_analysis.py:400:45 warning[possibly-unresolved-reference] Name `solver` used when possibly not defined
scipy (https://github.com/scipy/scipy)
- scipy/optimize/tests/test__numdiff.py:42:30 warning[possibly-unresolved-reference] Name `A` used when possibly not defined
- scipy/optimize/tests/test__numdiff.py:43:30 warning[possibly-unresolved-reference] Name `A` used when possibly not defined
- scipy/interpolate/tests/test_fitpack2.py:170:13 warning[possibly-unresolved-reference] Name `spl` used when possibly not defined
- scipy/interpolate/tests/test_interpolate.py:1229:62 warning[possibly-unresolved-reference] Name `x` used when possibly not defined
- scipy/io/arff/_arffread.py:449:12 warning[possibly-unresolved-reference] Name `restr` used when possibly not defined
- scipy/io/arff/_arffread.py:452:22 warning[possibly-unresolved-reference] Name `regexp` used when possibly not defined
- scipy/io/arff/_arffread.py:455:20 warning[possibly-unresolved-reference] Name `matches` used when possibly not defined
- scipy/io/arff/_arffread.py:456:13 warning[possibly-unresolved-reference] Name `matches` used when possibly not defined
- scipy/sparse/tests/test_base.py:1531:25 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/sparse/tests/test_base.py:1531:59 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/sparse/tests/test_base.py:1532:28 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/sparse/tests/test_base.py:1533:41 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/sparse/tests/test_base.py:1546:20 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/sparse/tests/test_base.py:1549:22 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/sparse/tests/test_base.py:1552:22 warning[possibly-unresolved-reference] Name `S_casted` used when possibly not defined
- scipy/spatial/transform/tests/test_rigid_transform.py:853:24 warning[possibly-unresolved-reference] Name `n` used when possibly not defined
spack (https://github.com/spack/spack)
- lib/spack/spack/package_prefs.py:149:27 warning[possibly-unresolved-reference] Name `variants` used when possibly not defined
- lib/spack/spack/package_prefs.py:150:33 warning[possibly-unresolved-reference] Name `variants` used when possibly not defined
- lib/spack/spack/package_prefs.py:154:46 warning[possibly-unresolved-reference] Name `variants` used when possibly not defined
- lib/spack/spack/package_prefs.py:225:8 warning[possibly-unresolved-reference] Name `readable` used when possibly not defined
- lib/spack/spack/package_prefs.py:227:8 warning[possibly-unresolved-reference] Name `readable` used when possibly not defined
- lib/spack/spack/package_prefs.py:230:8 warning[possibly-unresolved-reference] Name `writable` used when possibly not defined
- lib/spack/spack/package_prefs.py:231:12 warning[possibly-unresolved-reference] Name `readable` used when possibly not defined
- lib/spack/spack/package_prefs.py:238:8 warning[possibly-unresolved-reference] Name `writable` used when possibly not defined
- lib/spack/spack/package_prefs.py:239:12 warning[possibly-unresolved-reference] Name `readable` used when possibly not defined
- lib/spack/spack/package_prefs.py:261:12 warning[possibly-unresolved-reference] Name `group` used when possibly not defined
- lib/spack/spack/vendor/markupsafe/__init__.py:191:9 warning[possibly-unresolved-reference] Name `method` used when possibly not defined
- lib/spack/spack/vendor/ruamel/yaml/emitter.py:1512:20 warning[possibly-unresolved-reference] Name `pos` used when possibly not defined
spark (https://github.com/apache/spark)
- python/pyspark/ml/tests/connect/test_legacy_mode_evaluation.py:133:13 warning[possibly-unresolved-reference] Name `auprc_evaluator` used when possibly not defined
- python/pyspark/mllib/tests/test_feature.py:153:26 warning[possibly-unresolved-reference] Name `mat` used when possibly not defined
- python/pyspark/pandas/tests/frame/test_reindexing.py:550:42 warning[possibly-unresolved-reference] Name `keep` used when possibly not defined
- python/pyspark/pandas/tests/frame/test_reindexing.py:554:48 warning[possibly-unresolved-reference] Name `keep` used when possibly not defined
- python/pyspark/pandas/tests/frame/test_reindexing.py:551:43 warning[possibly-unresolved-reference] Name `keep` used when possibly not defined
- python/pyspark/pandas/tests/frame/test_reindexing.py:555:49 warning[possibly-unresolved-reference] Name `keep` used when possibly not defined
- python/pyspark/pandas/tests/groupby/test_aggregate.py:109:40 warning[possibly-unresolved-reference] Name `as_index` used when possibly not defined
- python/pyspark/sql/tests/test_functions.py:1586:59 warning[possibly-unresolved-reference] Name `aq` used when possibly not defined
static-frame (https://github.com/static-frame/static-frame)
- static_frame/test/property/test_util.py:259:25 warning[possibly-unresolved-reference] Name `post` used when possibly not defined
sympy (https://github.com/sympy/sympy)
- sympy/combinatorics/tests/test_perm_groups.py:195:5 warning[possibly-unresolved-reference] Name `G` used when possibly not defined
- sympy/combinatorics/tests/test_perm_groups.py:196:32 warning[possibly-unresolved-reference] Name `G` used when possibly not defined
- sympy/combinatorics/tests/test_perm_groups.py:196:35 warning[possibly-unresolved-reference] Name `G` used when possibly not defined
- sympy/concrete/tests/test_sums_products.py:792:12 warning[possibly-unresolved-reference] Name `func` used when possibly not defined
- sympy/concrete/tests/test_sums_products.py:793:12 warning[possibly-unresolved-reference] Name `func` used when possibly not defined
- sympy/concrete/tests/test_sums_products.py:794:12 warning[possibly-unresolved-reference] Name `func` used when possibly not defined
- sympy/concrete/tests/test_sums_products.py:795:12 warning[possibly-unresolved-reference] Name `func` used when possibly not defined
- sympy/series/residues.py:69:17 warning[possibly-unresolved-reference] Name `s` used when possibly not defined
- sympy/printing/pretty/pretty.py:1498:9 error[invalid-assignment] Object of type `Unknown` is not assignable to attribute `baseline` on type `None | Unknown | prettyForm`
- sympy/printing/pretty/pretty.py:1498:22 error[unresolved-attribute] Attribute `height` is not defined on `None` in union `None | Unknown | prettyForm`
- sympy/printing/pretty/pretty.py:1502:25 error[unresolved-attribute] Attribute `right` is not defined on `None` in union `None | Unknown | prettyForm`
- sympy/utilities/tests/test_iterables.py:416:39 warning[possibly-unresolved-reference] Name `nul` used when possibly not defined
- sympy/utilities/tests/test_iterables.py:418:39 warning[possibly-unresolved-reference] Name `nul` used when possibly not defined
- sympy/utilities/tests/test_iterables.py:419:39 warning[possibly-unresolved-reference] Name `nul` used when possibly not defined
tornado (https://github.com/tornadoweb/tornado)
- tornado/test/httpclient_test.py:682:34 warning[possibly-unresolved-reference] Name `resp` used when possibly not defined
vision (https://github.com/pytorch/vision)
- test/test_transforms.py:1949:26 warning[possibly-unresolved-reference] Name `mean` used when possibly not defined
- test/test_transforms.py:1949:32 warning[possibly-unresolved-reference] Name `std` used when possibly not defined
- torchvision/models/feature_extraction.py:601:87 warning[possibly-unresolved-reference] Name `name` used when possibly not defined9d3860e to
667a3fb
Compare
5b9c09e to
53006b1
Compare
53006b1 to
066e631
Compare
2ae6e9f to
1b3dc8e
Compare
add7d4b to
96af416
Compare
1c906d8 to
61346dc
Compare
61346dc to
7d2f48c
Compare
| { | ||
| let after_iter = self.flow_snapshot(); | ||
| let constraint = self | ||
| .record_reachability_constraint(PredicateOrLiteral::Literal(is_non_empty)); |
There was a problem hiding this comment.
I think this PR can be significantly simpler. There's no need to create a node in the reachability TDD for a reachability decision that we can make with full confidence immediately in semantic indexing. For these cases we can instead just merge or not-merge control flow snapshots directly here. I think the pattern here should be: "what is the known emptiness of the iterator? if empty, mark loop body unreachable and restore post-iterator snapshot before visiting else. if non-empty, visit body normally and do not merge zero-iteration path back in. if unknown, then we check for possibly adding a range constraint node, and proceed as main currently does."
There was a problem hiding this comment.
Good call, thank you!
cbd8135 to
5798505
Compare
5798505 to
4c6cc9b
Compare
|
|
||
| ```py | ||
| def _(items: list[int], mapping: dict[str, int]): | ||
| for item in [*items]: |
There was a problem hiding this comment.
It looks like we are smart about the fact that [*items, 1] is known non-empty (and similar for e.g. {*mapping, "c": 2}), but this is currently not tested.
4c6cc9b to
b9864b6
Compare
Summary
This extends the static reachability analysis for synchronous
forloops from #25220 to tuple, list, set, dictionary, string, and bytes literals whose emptiness is known syntactically. We record their static truthiness: a non-empty literal proves that the body executes at least once, while an empty literal marks the body unreachable and prevents its definitions from flowing past the loop.Starred elements and dictionary unpacking remain ambiguous when they are the only elements, while an explicit element still proves non-emptiness. Other iterables continue on the existing ambiguous path. Synchronous literals used in
async fordo not use this shortcut because they do not satisfy async iteration.