Fix package name normalization for package pypi json view by gerrod3 · Pull Request #717 · pulp/pulp_python

gerrod3 · 2024-08-08T03:14:49Z

fixes: #716

quba42 · 2024-08-08T07:16:43Z

I will try to test this patch today.

mdellweg · 2024-08-08T08:25:35Z

+    remote = python_remote_factory(includes=["oslo.utils"], url="https://pypi.org")
+    repo = python_repo_with_sync(remote)
+    distro = python_distribution_factory(repository=repo)
+    # Test pulp 2 pulp full sync w/ live pypi apis


I read pulp2. 🤦

mdellweg · 2024-08-08T08:27:36Z

+    pub = python_publication_factory(repository=repo)
+    distro2 = python_distribution_factory(publication=pub)
+    remote3 = python_remote_factory(includes=[], url=distro2.base_url)
+    repo3 = python_repo_with_sync(remote3)
+    summary2 = python_content_summary(repository_version=repo3.latest_version_href)


pulp_python can distribute with and without publications?
What is the difference? Can we deprecate and remove publications?

Yep, ever since I added the live api to support twine uploads publications were no longer necessary. Publications are still around to remain backwards compatible. The main difference is that publications are slightly more performant since we don't have to dynamically generate the simple files during pip installs. During pulp2pulp syncs it really is no different then the live api.

We probably could deprecate them, I would want to ask the community first if they would be okay with this. (Probably would also want to do some testing to see how much of a performance hit it would be).

mdellweg · 2024-08-08T08:30:12Z

+    remote2 = python_remote_factory(includes=[], url=distro.base_url)
+    repo2 = python_repo_with_sync(remote2)
+    summary = python_content_summary(repository_version=repo2.latest_version_href)
+    assert summary.present["python.python"]["count"] > 0


I'd love to see a bit more explanation here, why this assertion actually tells me that everything is ok. Maybe we need a short explanation of the bug that was fixed to describe that accurately.
Should we not also see the exact same number of content as in the repo here?

You're right we should see the same amount, forgot to add the check.

quba42 · 2024-08-08T09:07:18Z

I can confirm this code change fixes the issue in our Katello setup. I added the fix to both the Katello server and the proxy server for the Pulp to Pulp sync (not certain if it was needed on both ends), and now python packages with dot in the name also appear in the Pulp distribution on the proxy.

quba42 · 2024-08-08T09:12:00Z

@gerrod3 Any chance you can back-port this fix as far back as 3.11 or even 3.10 once it is reviewed and merged?

mdellweg · 2024-08-08T09:55:41Z

I see only 3.11 in the supported branches. Do we need to adjust that?

quba42 · 2024-08-08T11:51:45Z

I see only 3.11 in the supported branches. Do we need to adjust that?

I am going to say "no".

If it would have been trivial to backport to 3.10 I would have asked for it, but we mainly need 3.11 (used by Katello >= 4.13 with pulpcore 3.49). => I am happy with a back-port down to 3.11!

fixes: pulp#716

maximiliankolb · 2024-08-09T06:47:14Z

+    python_distribution_factory,
+    python_content_summary,
+):
+    """Test that Pulp can handle syncing packages with wierd names."""


Suggested change

"""Test that Pulp can handle syncing packages with wierd names."""

"""Test that Pulp can handle syncing packages with weird names."""

Or even more percise: "handle packages that contain dots"

patchback · 2024-08-13T12:33:18Z

Backport to 3.11: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply fc43b4b on top of patchback/backports/3.11/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717

Backporting merged PR #717 into main

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/pulp/pulp_python.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.11/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717 upstream/3.11

Now, cherry-pick PR Fix package name normalization for package pypi json view #717 contents into that branch:
```
$ git cherry-pick -x fc43b4b253dbec5931d6a230296942e4ba52ec3e
```
If it'll yell at you with something like fatal: Commit fc43b4b253dbec5931d6a230296942e4ba52ec3e is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x fc43b4b253dbec5931d6a230296942e4ba52ec3e
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Fix package name normalization for package pypi json view #717 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.11/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2024-08-13T12:33:18Z

Backport to 3.12: 💚 backport PR created

✅ Backport PR branch: patchback/backports/3.12/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717

Backported as #725

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

…dbec5931d6a230296942e4ba52ec3e/pr-717 [PR #717/fc43b4b2 backport][3.12] Fix package name normalization for package pypi json view

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

Merge pull request pulp#717 from gerrod3/normalize-name-pypi (cherry picked from commit fc43b4b)

…253dbec5931d6a230296942e4ba52ec3e/pr-717 Backport #717 to 3.11

patchback · 2025-07-14T12:30:47Z

Backport to 3.10: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply fc43b4b on top of patchback/backports/3.10/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717

Backporting merged PR #717 into main

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/pulp/pulp_python.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.10/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717 upstream/3.10

Now, cherry-pick PR Fix package name normalization for package pypi json view #717 contents into that branch:
```
$ git cherry-pick -x fc43b4b253dbec5931d6a230296942e4ba52ec3e
```
If it'll yell at you with something like fatal: Commit fc43b4b253dbec5931d6a230296942e4ba52ec3e is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x fc43b4b253dbec5931d6a230296942e4ba52ec3e
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Fix package name normalization for package pypi json view #717 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.10/fc43b4b253dbec5931d6a230296942e4ba52ec3e/pr-717

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

…b253dbec5931d6a230296942e4ba52ec3e/pr-717 [PR #717/fc43b4b2 backport][3.10] Fix package name normalization for package pypi json view

mdellweg reviewed Aug 8, 2024

View reviewed changes

Fix package name normalization for package pypi json view

4880949

fixes: pulp#716

gerrod3 force-pushed the normalize-name-pypi branch from 8a670b8 to 4880949 Compare August 8, 2024 16:54

maximiliankolb reviewed Aug 9, 2024

View reviewed changes

mdellweg approved these changes Aug 9, 2024

View reviewed changes

gerrod3 merged commit fc43b4b into pulp:main Aug 13, 2024

gerrod3 added backport-3.11 backport-3.12 labels Aug 13, 2024

patchback Bot pushed a commit that referenced this pull request Aug 13, 2024

Merge pull request #717 from gerrod3/normalize-name-pypi

ec7b1ac

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

patchback Bot mentioned this pull request Aug 13, 2024

[PR #717/fc43b4b2 backport][3.12] Fix package name normalization for package pypi json view #725

Merged

mdellweg added a commit that referenced this pull request Aug 13, 2024

Merge pull request #725 from pulp/patchback/backports/3.12/fc43b4b253…

8a476ac

…dbec5931d6a230296942e4ba52ec3e/pr-717 [PR #717/fc43b4b2 backport][3.12] Fix package name normalization for package pypi json view

gerrod3 deleted the normalize-name-pypi branch August 19, 2024 13:22

gerrod3 added a commit to gerrod3/pulp_python that referenced this pull request Aug 19, 2024

Merge pull request pulp#717 from gerrod3/normalize-name-pypi

8a57fcd

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

gerrod3 added a commit to gerrod3/pulp_python that referenced this pull request Aug 19, 2024

Fix package name normalization for package pypi json view

957f60c

Merge pull request pulp#717 from gerrod3/normalize-name-pypi (cherry picked from commit fc43b4b)

gerrod3 added a commit that referenced this pull request Aug 21, 2024

Merge pull request #731 from gerrod3/patchback/backports/3.11/fc43b4b…

1ba3c5c

…253dbec5931d6a230296942e4ba52ec3e/pr-717 Backport #717 to 3.11

jobselko added the backport-3.10 label Jul 14, 2025

jobselko pushed a commit to jobselko/pulp_python that referenced this pull request Jul 14, 2025

Merge pull request pulp#717 from gerrod3/normalize-name-pypi

228fb43

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

jobselko pushed a commit to jobselko/pulp_python that referenced this pull request Jul 14, 2025

Merge pull request pulp#717 from gerrod3/normalize-name-pypi

75aaaef

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

jobselko mentioned this pull request Jul 15, 2025

[PR #717/fc43b4b2 backport][3.10] Fix package name normalization for package pypi json view #905

Merged

jobselko pushed a commit to jobselko/pulp_python that referenced this pull request Jul 15, 2025

Merge pull request pulp#717 from gerrod3/normalize-name-pypi

3f0cc02

Fix package name normalization for package pypi json view (cherry picked from commit fc43b4b)

jobselko added a commit that referenced this pull request Jul 18, 2025

Merge pull request #905 from jobselko/patchback/backports/3.10/fc43b4…

a5b9256

…b253dbec5931d6a230296942e4ba52ec3e/pr-717 [PR #717/fc43b4b2 backport][3.10] Fix package name normalization for package pypi json view

	"""Test that Pulp can handle syncing packages with wierd names."""
	"""Test that Pulp can handle syncing packages with weird names."""

Conversation

gerrod3 commented Aug 8, 2024

Uh oh!

quba42 commented Aug 8, 2024

Uh oh!

mdellweg Aug 8, 2024

Choose a reason for hiding this comment

Uh oh!

mdellweg Aug 8, 2024

Choose a reason for hiding this comment

Uh oh!

gerrod3 Aug 8, 2024

Choose a reason for hiding this comment

Uh oh!

mdellweg Aug 8, 2024

Choose a reason for hiding this comment

Uh oh!

gerrod3 Aug 8, 2024

Choose a reason for hiding this comment

Uh oh!

quba42 commented Aug 8, 2024

Uh oh!

quba42 commented Aug 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mdellweg commented Aug 8, 2024

Uh oh!

quba42 commented Aug 8, 2024

Uh oh!

maximiliankolb Aug 9, 2024

Choose a reason for hiding this comment

Uh oh!

patchback Bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 3.11: 💔 cherry-picking failed — conflicts found

Backporting merged PR #717 into main

Uh oh!

patchback Bot commented Aug 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 3.12: 💚 backport PR created

Uh oh!

patchback Bot commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 3.10: 💔 cherry-picking failed — conflicts found

Backporting merged PR #717 into main

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

quba42 commented Aug 8, 2024 •

edited

Loading

patchback Bot commented Aug 13, 2024 •

edited

Loading

patchback Bot commented Aug 13, 2024 •

edited

Loading

patchback Bot commented Jul 14, 2025 •

edited

Loading