Skip to content

[WIP] PEP-514: Integrates feedback from python-dev.#59

Merged
zooba merged 5 commits into
python:masterfrom
zooba:pep-514
Jul 23, 2016
Merged

[WIP] PEP-514: Integrates feedback from python-dev.#59
zooba merged 5 commits into
python:masterfrom
zooba:pep-514

Conversation

@zooba

@zooba zooba commented Jul 18, 2016

Copy link
Copy Markdown
Member

No description provided.

@brettcannon brettcannon changed the title PEP-514: Integrates feedback from python-dev. [WIP] PEP-514: Integrates feedback from python-dev. Jul 18, 2016
Comment thread pep-0514.txt
installers to register their installation, and to allow tools and applications
to detect and correctly display all Python environments on a user's machine. No
implementation changes to Python are proposed with this PEP.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still struggling to understand what (in general) tools that consume this metadata would look like. There's a lack of motivating examples here. The only examples I can think of are virtualenv and "something that lists all the available installations" (but I'm not clear what it's then going to do with the user's selection). You mention Visual Studio as another consumer of this data. It would be really useful to have a description of which data VS uses, and how it exposes it to the user. And any other motivating cases you're aware of. The other use case I can think of, unfortunately obsolete, is the bdist_wininst installer, which presented a "which Python installation do you want" dialog. But I don't recall the details of that, and it's basically obsolete now anyway. As the PEP stands, it feels like a collection of things that we think would be "nice to have", but we're not sure enough of ourselves to make them mandatory.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a few screenshots of Visual Studio here, a sample of some of the extra info being shown here. We use everything in the PEP, since I fleshed out the PEP based on what we wanted to use (mostly - there are a few things we didn't bother with, such as localized text). The other IDEs have similar functionality, I just don't have handy screenshots of it..

FWIW, >90% of Windows users of Python don't seem to be using the command line ever...

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks. Maybe add those links as footnotes in the PEP?

In that "add an environment" dialog, how much is required? I ask because I'd argue that if any of the fields are mandatory, then the corresponding data item in the PEP should probably be mandatory (not "unknown").

I'm not sure about your >90% figure, we get a lot of bug reports for pip and virtualenv from Windows users. I can't imagine those typically come from IDE users. Like everything, I suspect it depends on what particular interest area you deal with, though.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe 90% is too many, but there's a big iceberg of Windows users who have their own internal support teams, aren't using pip/virtualenv, or aren't allowed to comment publicly on anything even to report bugs. And I believe all the IDEs just act as thin wrappers around those tools and will pass bug reports on.

None of the information in that dialog is required, but if parts are missing you'll get a substandard experience. For example, we base some editor features on the language version, so if you pick the wrong one, you'll get the wrong squiggles/etc. If we don't have an executable, we can't run it. They're all fairly obvious IMO.

@jayvdb

jayvdb commented Jul 19, 2016

Copy link
Copy Markdown

For the values of some of the keys, it would make sense to map them to PEP 496 identifiers. It would be odd if the values in the registry differed from the equivalent in the pep about environment markers.

If the registry had a standardised list of key names for all environment markers, non-python (native) apps could implement pep 496 mini language to download and install an appropriate wheel into a user selected environment that provided values for those keys, without needing to invoke the python interpreter for that environment. It might also need to know where to put the package, but that can be derived from platform_python_implementation and specified in a separate pep.

Rather than use virtualenv to think about this pep as an example use, even implicitly, a better example might be pyenv which attempts to do the same without invoking python wherever possible. Theoretically it could be ported to a bash on windows with a few gnu binaries added.

@pfmoore

pfmoore commented Jul 19, 2016

Copy link
Copy Markdown
Member

If the registry had a standardised list of key names for all environment markers, non-python (native) apps could implement pep 496 mini language to download and install an appropriate wheel into a user selected environment that provided values for those keys, without needing to invoke the python interpreter for that environment.

Thanks. This is a good example of a non-IDE use case.

a better example might be pyenv

I guess the key point here is pyenv versions which lists the available versions to select from. I don't know where the Unix pyenv gets its version names from, but the list shown in the example screenshot on the page you linked to:

2.7.10
3.5.0
miniconda3-3.16.0
pypy-2.6.0

is the sort of naming I'd expect in a command line utility (as opposed to the ExampleCorp\distro-3 style, with no explicit version, that the PEP is currently implying).

@jayvdb

jayvdb commented Jul 19, 2016

Copy link
Copy Markdown

The names are defined by the filenames in https://github.com/yyuu/pyenv/tree/master/plugins/python-build/share/python-build

@pfmoore

pfmoore commented Jul 19, 2016

Copy link
Copy Markdown
Member

OK, cool, so by pyvenv itself. That's similar to my thought about a "short alias to Company/Key mapping", and so tends to imply that it's not actually unreasonable for tools to manage their own short names.

I'm becoming more and more inclined not to worry about getting hung up on the format of "Company" and "Key", beyond saying that we recommend human-readable, short, descriptive values, and I'd like to establish the principle that Key should typically be a version. Ideally the PythonCore values should provide the example to follow - I'm just a little concerned that the PEP examples imply "anything goes" a bit too much.

@zooba

zooba commented Jul 19, 2016

Copy link
Copy Markdown
Member Author

I would rather Key not include a specific version unless it can be installed side-by-side with others. e.g. miniconda3 is sufficient, because nobody should have miniconda3-3.16.0 and miniconda3-3.17.0 simultaneously. We don't have PythonCore\3.5.1 and PythonCore\3.5.2 - it's just 3.5.

Not sure if it was missed (or if I edited it out), but I suggested at one point that tools like pyenv/virtualenv should probably allow use of just the Tag to identify an installation when it's unambiguous. So given:

PythonCore\2.7
PythonCore\3.5
WinPython\3.5
Continuum\anaconda3

You could do:

virtualenv -p 2.7
# success!

virtualenv -p anaconda3
# success!

virtualenv -p 3.5
# failure, probably with a printout of
Did you mean:
  PythonCore\3.5
  WinPython\3.5

I could add this as an example, but I'm really hesitant to add anything that appears to dictate UI. If a tool wants to resolve in some other way, that's up to them and their users (bearing in mind that most tools will be internal to various companies and never see the light of day).

Comment thread pep-0514.txt Outdated
installations.
installations. For example, ``virtualenv`` supports a ``-p`` argument to use an
alternative interpreter, but is unable to provide a list of all values that may
be provided by a user.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed that this hypothetical motivation isn't as motivating as the fact that py.exe and some IDEs are already trusting the information, despite the potential for collisions. So I'm changing this around slightly to emphasise that we're fixing a problem here, not adding a new feature.

@zooba

zooba commented Jul 19, 2016

Copy link
Copy Markdown
Member Author

I'm +0 right now on adding an "Illustrative Examples" section, both with some registry layouts and code for reading them (and also a GetBinaryType call to handle 32-bit vs 64-bit detection without running the interpreter).

My main hesitation is that I don't want examples to become binding (i.e. "you're only valid if the code in the PEP displays everything"), especially since (by design), practically everything in the PEP is optional.

@pfmoore

pfmoore commented Jul 19, 2016

Copy link
Copy Markdown
Member

OK. Thanks for the updates and clarifications. I think they help a lot.

I don't honestly think there's much problem with adding examples, and I think it would help a lot. By all means add a disclaimer that the example code is for illustration only. I do think that an example should cater for missing data (run the interpreter -> look for the ExecutablePath value, if not, try InstallPath\python.exe, but allow for that exe not to exist), as IMO an example that shows what proper error handling looks like will stop people from making unwarranted assumptions in their production code. (If the resulting example is ugly and over-complex, that may say something about the spec, of course :-)) Personally, I'd like to see some code in here.

At the moment, I'd characterise this PEP as doing a few things:

  1. Giving a clear message to non-python.org distributions that they should not use our registry space, and providing an alternative for them,
  2. Agreeing the basic Company\Key structure for finding sys.prefix.
  3. Suggesting some informational data that distributions can optionally supply, and tools can optionally use (the intended use being for information display, likely in a GUI).
  4. Providing a little more detail on historically how the python.org installers used the registry.

Within that scope, I'm happy with the PEP. I think I initially expected more from it - and I still think it's a shame that it doesn't go further in terms of "introspection without needing to run the interpreter" - but it achieves the above goals, which is great.

If you can decide on whether you're willing to add the example code, then I think we're there. I'll post a last call for comments on python-dev, for people who haven't been following the tracker discussion, and then we can sign this off.

Adds ExecutableArguments and WindowedExecutableArguments for fixed launch arguments.
Adds first (of multiple) examples
@zooba

zooba commented Jul 23, 2016

Copy link
Copy Markdown
Member Author

Added hopefully the final textual clarifications and the first example. It's a shame that dealing with the registry is so messy in Python - there's far more indentation there than I'd like, and it doesn't even include the backwards compatibility handling or most of the metadata.

Will try and get to a couple more examples over the coming week, but wanted to push it online in case I change machine before I next get a chance to work on it.

@pfmoore

pfmoore commented Jul 23, 2016

Copy link
Copy Markdown
Member

Thanks! This looks pretty good now. Once you've completed the changes you want to make, I'm happy to sign this off (on a procedural note, I'm not sure if it needs a posting of the final version to python-dev, or if we can just refer back to here. I'll check on that.)

@zooba

zooba commented Jul 23, 2016

Copy link
Copy Markdown
Member Author

I just finished another couple of examples, including the values that will be written for a 3.6 install (once the PEP is accepted and I update the installer, that is).

I think we should merge the changes and post one more time on python-dev. That way we'll have the mailing list trail of acceptance and I can do the final update here with the usual links.

@pfmoore

pfmoore commented Jul 23, 2016

Copy link
Copy Markdown
Member

Sounds fine to me. Let's do that and unless anyone comes up with a last-minute objection (!), I'll formally accept the PEP on python-dev.

I'm also in the process of posting a note to python-committers suggesting that we update PEP 1 to cover this process, so it's formally documented.

@zooba zooba merged commit c9f126a into python:master Jul 23, 2016
@zooba zooba deleted the pep-514 branch July 23, 2016 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants