[Python-Dev] PEP 207 -- Rich Comparisons
Greg Wilson
gvwilson@nevex.com
Fri, 15 Dec 2000 16:43:47 -0500
This is a multi-part message in MIME format.
------=_NextPart_000_002A_01C066B6.32690BC0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Hi, Paul; thanks for your mail. W.r.t. adding matrix operators to Python,
you may want to take a look at the counter-arguments in PEP 0211 (attached).
Basically, I spoke with the authors of GNU Octave (a GPL'd clone of MATLAB)
about what users really used. They felt that the only matrix operator that
really mattered was matrix-matrix multiply; other operators (including the
left and right division operators that even experienced MATLAB users often
mix up) were second order at best, and were better handled with methods or
functions.
Thanks,
Greg
p.s. PEP 0225 (also attached) is an alternative to PEP 0211 which would add
most of the MATLAB-ish operators to Python.
------=_NextPart_000_002A_01C066B6.32690BC0
Content-Type: text/plain;
name="pep-0211.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="pep-0211.txt"
PEP: 211=0A=
Title: Adding New Linear Algebra Operators to Python=0A=
Version: $Revision: 1.5 $=0A=
Author: gvwilson@nevex.com (Greg Wilson)=0A=
Status: Draft=0A=
Type: Standards Track=0A=
Python-Version: 2.1=0A=
Created: 15-Jul-2000=0A=
Post-History:=0A=
=0A=
=0A=
Introduction=0A=
=0A=
This PEP describes a conservative proposal to add linear algebra=0A=
operators to Python 2.0. It discusses why such operators are=0A=
desirable, and why a minimalist approach should be adopted at this=0A=
point. This PEP summarizes discussions held in mailing list=0A=
forums, and provides URLs for further information, where=0A=
appropriate. The CVS revision history of this file contains the=0A=
definitive historical record.=0A=
=0A=
=0A=
Summary=0A=
=0A=
Add a single new infix binary operator '@' ("across"), and=0A=
corresponding special methods "__across__()", "__racross__()", and=0A=
"__iacross__()". This operator will perform mathematical matrix=0A=
multiplication on NumPy arrays, and generate cross-products when=0A=
applied to built-in sequence types. No existing operator=0A=
definitions will be changed.=0A=
=0A=
=0A=
Background=0A=
=0A=
The first high-level programming language, Fortran, was invented=0A=
to do arithmetic. While this is now just a small part of=0A=
computing, there are still many programmers who need to express=0A=
complex mathematical operations in code.=0A=
=0A=
The most influential of Fortran's successors was APL [1]. Its=0A=
author, Kenneth Iverson, designed the language as a notation for=0A=
expressing matrix algebra, and received the 1980 Turing Award for=0A=
his work.=0A=
=0A=
APL's operators supported both familiar algebraic operations, such=0A=
as vector dot product and matrix multiplication, and a wide range=0A=
of structural operations, such as stitching vectors together to=0A=
create arrays. Even by programming's standards, APL is=0A=
exceptionally cryptic: many of its symbols did not exist on=0A=
standard keyboards, and expressions have to be read right to left.=0A=
=0A=
Most subsequent work numerical languages, such as Fortran-90,=0A=
MATLAB, and Mathematica, have tried to provide the power of APL=0A=
without the obscurity. Python's NumPy [2] has most of the=0A=
features that users of such languages expect, but these are=0A=
provided through named functions and methods, rather than=0A=
overloaded operators. This makes NumPy clumsier than most=0A=
alternatives.=0A=
=0A=
The author of this PEP therefore consulted the developers of GNU=0A=
Octave [3], an open source clone of MATLAB. When asked how=0A=
important it was to have infix operators for matrix solution,=0A=
Prof. James Rawlings replied [4]:=0A=
=0A=
I DON'T think it's a must have, and I do a lot of matrix=0A=
inversion. I cannot remember if its A\b or b\A so I always=0A=
write inv(A)*b instead. I recommend dropping \.=0A=
=0A=
Rawlings' feedback on other operators was similar. It is worth=0A=
noting in this context that notations such as "/" and "\" for=0A=
matrix solution were invented by programmers, not mathematicians,=0A=
and have not been adopted by the latter.=0A=
=0A=
Based on this discussion, and feedback from classes at the US=0A=
national laboratories and elsewhere, we recommend only adding a=0A=
matrix multiplication operator to Python at this time. If there=0A=
is significant user demand for syntactic support for other=0A=
operations, these can be added in a later release.=0A=
=0A=
=0A=
Requirements=0A=
=0A=
The most important requirement is minimal impact on existing=0A=
Python programs and users: the proposal must not break existing=0A=
code (except possibly NumPy).=0A=
=0A=
The second most important requirement is the ability to handle all=0A=
common cases cleanly and clearly. There are nine such cases:=0A=
=0A=
|5 6| * 9 =3D |45 54| MS: matrix-scalar multiplication=0A=
|7 8| |63 72|=0A=
=0A=
9 * |5 6| =3D |45 54| SM: scalar-matrix multiplication=0A=
|7 8| |63 72|=0A=
=0A=
|2 3| * |4 5| =3D |8 15| VE: vector elementwise =
multiplication=0A=
=0A=
=0A=
|2 3| * |4| =3D 23 VD: vector dot product=0A=
|5|=0A=
=0A=
|2| * |4 5| =3D | 8 10| VO: vector outer product=0A=
|3| |12 15|=0A=
=0A=
|1 2| * |5 6| =3D | 5 12| ME: matrix elementwise =
multiplication=0A=
|3 4| |7 8| |21 32|=0A=
=0A=
|1 2| * |5 6| =3D |19 22| MM: mathematical matrix =
multiplication=0A=
|3 4| |7 8| |43 50|=0A=
=0A=
|1 2| * |5 6| =3D |19 22| VM: vector-matrix multiplication=0A=
|7 8|=0A=
=0A=
|5 6| * |1| =3D |17| MV: matrix-vector multiplication=0A=
|7 8| |2| |23|=0A=
=0A=
Note that 1-dimensional vectors are treated as rows in VM, as=0A=
columns in MV, and as both in VD and VO. Both are special cases=0A=
of 2-dimensional matrices (Nx1 and 1xN respectively). We will=0A=
therefore define the new operator only for 2-dimensional arrays,=0A=
and provide an easy (and efficient) way for users to treat=0A=
1-dimensional structures as 2-dimensional.=0A=
=0A=
Third, we must avoid confusion between Python's notation and those=0A=
of MATLAB and Fortran-90. In particular, mathematical matrix=0A=
multiplication (case MM) should not be represented as '.*', since:=0A=
=0A=
(a) MATLAB uses prefix-'.' forms to mean 'elementwise', and raw=0A=
forms to mean "mathematical"; and=0A=
=0A=
(b) even if the Python parser can be taught how to handle dotted=0A=
forms, '1.*A' will still be visually ambiguous.=0A=
=0A=
=0A=
Proposal=0A=
=0A=
The meanings of all existing operators will be unchanged. In=0A=
particular, 'A*B' will continue to be interpreted elementwise.=0A=
This takes care of the cases MS, SM, VE, and ME, and ensures=0A=
minimal impact on existing programs.=0A=
=0A=
A new operator '@' (pronounced "across") will be added to Python,=0A=
along with special methods "__across__()", "__racross__()", and=0A=
"__iacross__()", with the usual semantics. (We recommend using=0A=
"@", rather than the times-like "><", because of the ease with=0A=
which the latter could be mis-typed as inequality "<>".)=0A=
=0A=
No new operators will be defined to mean "solve a set of linear=0A=
equations", or "invert a matrix".=0A=
=0A=
(Optional) When applied to sequences, the "@" operator will return=0A=
a tuple of tuples containing the cross-product of their elements=0A=
in left-to-right order:=0A=
=0A=
>>> [1, 2] @ (3, 4)=0A=
((1, 3), (1, 4), (2, 3), (2, 4))=0A=
=0A=
>>> [1, 2] @ (3, 4) @ (5, 6)=0A=
((1, 3, 5), (1, 3, 6), =0A=
(1, 4, 5), (1, 4, 6),=0A=
(2, 3, 5), (2, 3, 6),=0A=
(2, 4, 5), (2, 4, 6))=0A=
=0A=
This will require the same kind of special support from the parser=0A=
as chained comparisons (such as "a<b<c<=3Dd"). However, it will=0A=
permit:=0A=
=0A=
>>> for (i, j) in [1, 2] @ [3, 4]:=0A=
>>> print i, j=0A=
1 3=0A=
1 4=0A=
2 3=0A=
2 4=0A=
=0A=
as a short-hand for the common nested loop idiom:=0A=
=0A=
>>> for i in [1, 2]:=0A=
>>> for j in [3, 4]:=0A=
>>> print i, j=0A=
=0A=
Response to the 'lockstep loop' questionnaire [5] indicated that=0A=
newcomers would be comfortable with this (so comfortable, in fact,=0A=
that most of them interpreted most multi-loop 'zip' syntaxes [6]=0A=
as implementing single-stage nesting).=0A=
=0A=
=0A=
Alternatives=0A=
=0A=
01. Don't add new operators.=0A=
=0A=
Python is not primarily a numerical language; it may not be worth=0A=
complexifying it for this special case. NumPy's success is proof=0A=
that users can and will use functions and methods for linear=0A=
algebra. However, support for real matrix multiplication is=0A=
frequently requested, as:=0A=
=0A=
* functional forms are cumbersome for lengthy formulas, and do not=0A=
respect the operator precedence rules of conventional mathematics;=0A=
and=0A=
=0A=
* method forms are asymmetric in their operands.=0A=
=0A=
What's more, the proposed semantics for "@" for built-in sequence=0A=
types would simplify expression of a very common idiom (nested=0A=
loops). User testing during discussion of 'lockstep loops'=0A=
indicated that both new and experienced users would understand=0A=
this immediately.=0A=
=0A=
02. Introduce prefixed forms of all existing operators, such as=0A=
"~*" and "~+", as proposed in PEP 0225 [7].=0A=
=0A=
This proposal would duplicate all built-in mathematical operators=0A=
with matrix equivalents, as in numerical languages such as=0A=
MATLAB. Our objections to this are:=0A=
=0A=
* Python is not primarily a numerical programming language. While=0A=
the (self-selected) participants in the discussions that led to=0A=
PEP 0225 may want all of these new operators, the majority of=0A=
Python users would be indifferent. The extra complexity they=0A=
would introduce into the language therefore does not seem=0A=
merited. (See also Rawlings' comments, quoted in the Background=0A=
section, about these operators not being essential.)=0A=
=0A=
* The proposed syntax is difficult to read (i.e. passes the "low=0A=
toner" readability test).=0A=
=0A=
03. Retain the existing meaning of all operators, but create a=0A=
behavioral accessor for arrays, such that:=0A=
=0A=
A * B=0A=
=0A=
is elementwise multiplication (ME), but:=0A=
=0A=
A.m() * B.m()=0A=
=0A=
is mathematical multiplication (MM). The method "A.m()" would=0A=
return an object that aliased A's memory (for efficiency), but=0A=
which had a different implementation of __mul__().=0A=
=0A=
This proposal was made by Moshe Zadka, and is also considered by=0A=
PEP 0225 [7]. Its advantage is that it has no effect on the=0A=
existing implementation of Python: changes are localized in the=0A=
Numeric module. The disadvantages are=0A=
=0A=
* The semantics of "A.m() * B", "A + B.m()", and so on would have=0A=
to be defined, and there is no "obvious" choice for them.=0A=
=0A=
* Aliasing objects to trigger different operator behavior feels=0A=
less Pythonic than either calling methods (as in the existing=0A=
Numeric module) or using a different operator. This PEP is=0A=
primarily about look and feel, and about making Python more=0A=
attractive to people who are not already using it.=0A=
=0A=
=0A=
Related Proposals=0A=
=0A=
0207 : Rich Comparisons=0A=
=0A=
It may become possible to overload comparison operators=0A=
such as '<' so that an expression such as 'A < B' returns=0A=
an array, rather than a scalar value.=0A=
=0A=
0209 : Adding Multidimensional Arrays=0A=
=0A=
Multidimensional arrays are currently an extension to=0A=
Python, rather than a built-in type.=0A=
=0A=
0225 : Elementwise/Objectwise Operators=0A=
=0A=
A larger proposal that addresses the same subject, but=0A=
which proposes many more additions to the language.=0A=
=0A=
=0A=
Acknowledgments=0A=
=0A=
I am grateful to Huaiyu Zhu [8] for initiating this discussion,=0A=
and for some of the ideas and terminology included below.=0A=
=0A=
=0A=
References=0A=
=0A=
[1] http://www.acm.org/sigapl/whyapl.htm=0A=
[2] http://numpy.sourceforge.net=0A=
[3] http://bevo.che.wisc.edu/octave/=0A=
[4] http://www.egroups.com/message/python-numeric/4=0A=
[5] http://www.python.org/pipermail/python-dev/2000-July/013139.html=0A=
[6] PEP-0201.txt "Lockstep Iteration"=0A=
[7] =
http://www.python.org/pipermail/python-list/2000-August/112529.html=0A=
=0A=
=0A=
Appendix: NumPy=0A=
=0A=
NumPy will overload "@" to perform mathematical multiplication of=0A=
arrays where shapes permit, and to throw an exception otherwise.=0A=
Its implementation of "@" will treat built-in sequence types as if=0A=
they were column vectors. This takes care of the cases MM and MV.=0A=
=0A=
An attribute "T" will be added to the NumPy array type, such that=0A=
"m.T" is:=0A=
=0A=
(a) the transpose of "m" for a 2-dimensional array=0A=
=0A=
(b) the 1xN matrix transpose of "m" if "m" is a 1-dimensional=0A=
array; or=0A=
=0A=
(c) a runtime error for an array with rank >=3D 3.=0A=
=0A=
This attribute will alias the memory of the base object. NumPy's=0A=
"transpose()" function will be extended to turn built-in sequence=0A=
types into row vectors. This takes care of the VM, VD, and VO=0A=
cases. We propose an attribute because:=0A=
=0A=
(a) the resulting notation is similar to the 'superscript T' (at=0A=
least, as similar as ASCII allows), and=0A=
=0A=
(b) it signals that the transposition aliases the original object.=0A=
=0A=
NumPy will define a value "inv", which will be recognized by the=0A=
exponentiation operator, such that "A ** inv" is the inverse of=0A=
"A". This is similar in spirit to NumPy's existing "newaxis"=0A=
value.=0A=
=0A=
=0A=
=0C=0A=
Local Variables:=0A=
mode: indented-text=0A=
indent-tabs-mode: nil=0A=
End:=0A=
------=_NextPart_000_002A_01C066B6.32690BC0
Content-Type: text/plain;
name="pep-0225.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="pep-0225.txt"
PEP: 225=0A=
Title: Elementwise/Objectwise Operators=0A=
Version: $Revision: 1.2 $=0A=
Author: hzhu@users.sourceforge.net (Huaiyu Zhu),=0A=
gregory.lielens@fft.be (Gregory Lielens)=0A=
Status: Draft =0A=
Type: Standards Track=0A=
Python-Version: 2.1=0A=
Created: 19-Sep-2000=0A=
Post-History:=0A=
=0A=
=0A=
Introduction=0A=
=0A=
This PEP describes a proposal to add new operators to Python which=0A=
are useful for distinguishing elementwise and objectwise=0A=
operations, and summarizes discussions in the news group=0A=
comp.lang.python on this topic. See Credits and Archives section=0A=
at end. Issues discussed here include:=0A=
=0A=
- Background.=0A=
- Description of proposed operators and implementation issues.=0A=
- Analysis of alternatives to new operators.=0A=
- Analysis of alternative forms.=0A=
- Compatibility issues=0A=
- Description of wider extensions and other related ideas.=0A=
=0A=
A substantial portion of this PEP describes ideas that do not go=0A=
into the proposed extension. They are presented because the=0A=
extension is essentially syntactic sugar, so its adoption must be=0A=
weighed against various possible alternatives. While many=0A=
alternatives may be better in some aspects, the current proposal=0A=
appears to be overall advantageous.=0A=
=0A=
The issues concerning elementwise-objectwise operations extends to=0A=
wider areas than numerical computation. This document also=0A=
describes how the current proposal may be integrated with more=0A=
general future extensions.=0A=
=0A=
=0A=
Background=0A=
=0A=
Python provides six binary infix math operators: + - * / % **=0A=
hereafter generically represented by "op". They can be overloaded=0A=
with new semantics for user-defined classes. However, for objects=0A=
composed of homogeneous elements, such as arrays, vectors and=0A=
matrices in numerical computation, there are two essentially=0A=
distinct flavors of semantics. The objectwise operations treat=0A=
these objects as points in multidimensional spaces. The=0A=
elementwise operations treat them as collections of individual=0A=
elements. These two flavors of operations are often intermixed in=0A=
the same formulas, thereby requiring syntactical distinction.=0A=
=0A=
Many numerical computation languages provide two sets of math=0A=
operators. For example, in MatLab, the ordinary op is used for=0A=
objectwise operation while .op is used for elementwise operation.=0A=
In R, op stands for elementwise operation while %op% stands for=0A=
objectwise operation.=0A=
=0A=
In Python, there are other methods of representation, some of=0A=
which already used by available numerical packages, such as=0A=
=0A=
- function: mul(a,b)=0A=
- method: a.mul(b)=0A=
- casting: a.E*b =0A=
=0A=
In several aspects these are not as adequate as infix operators.=0A=
More details will be shown later, but the key points are=0A=
=0A=
- Readability: Even for moderately complicated formulas, infix=0A=
operators are much cleaner than alternatives.=0A=
=0A=
- Familiarity: Users are familiar with ordinary math operators.=0A=
=0A=
- Implementation: New infix operators will not unduly clutter=0A=
Python syntax. They will greatly ease the implementation of=0A=
numerical packages.=0A=
=0A=
While it is possible to assign current math operators to one=0A=
flavor of semantics, there is simply not enough infix operators to=0A=
overload for the other flavor. It is also impossible to maintain=0A=
visual symmetry between these two flavors if one of them does not=0A=
contain symbols for ordinary math operators.=0A=
=0A=
=0A=
Proposed extension=0A=
=0A=
- Six new binary infix operators ~+ ~- ~* ~/ ~% ~** are added to=0A=
core Python. They parallel the existing operators + - * / % **.=0A=
=0A=
- Six augmented assignment operators ~+=3D ~-=3D ~*=3D ~/=3D ~%=3D =
~**=3D are=0A=
added to core Python. They parallel the operators +=3D -=3D *=3D =
/=3D=0A=
%=3D **=3D available in Python 2.0.=0A=
=0A=
- Operator ~op retains the syntactical properties of operator op,=0A=
including precedence.=0A=
=0A=
- Operator ~op retains the semantical properties of operator op on=0A=
built-in number types.=0A=
=0A=
- Operator ~op raise syntax error on non-number builtin types.=0A=
This is temporary until the proper behavior can be agreed upon.=0A=
=0A=
- These operators are overloadable in classes with names that=0A=
prepend "t" (for tilde) to names of ordinary math operators.=0A=
For example, __tadd__ and __rtadd__ work for ~+ just as __add__=0A=
and __radd__ work for +.=0A=
=0A=
- As with exiting operators, the __r*__() methods are invoked when=0A=
the left operand does not provide the appropriate method.=0A=
=0A=
It is intended that one set of op or ~op is used for elementwise=0A=
operations, the other for objectwise operations, but it is not=0A=
specified which version of operators stands for elementwise or=0A=
objectwise operations, leaving the decision to applications.=0A=
=0A=
The proposed implementation is to patch several files relating to=0A=
the tokenizer, parser, grammar and compiler to duplicate the=0A=
functionality of corresponding existing operators as necessary.=0A=
All new semantics are to be implemented in the classes that=0A=
overload them.=0A=
=0A=
The symbol ~ is already used in Python as the unary "bitwise not"=0A=
operator. Currently it is not allowed for binary operators. The=0A=
new operators are completely backward compatible.=0A=
=0A=
=0A=
Prototype Implementation=0A=
=0A=
Greg Lielens implemented the infix ~op as a patch against Python=0A=
2.0b1 source[1].=0A=
=0A=
To allow ~ to be part of binary operators, the tokenizer would=0A=
treat ~+ as one token. This means that currently valid expression=0A=
~+1 would be tokenized as ~+ 1 instead of ~ + 1. The parser would=0A=
then treat ~+ as composite of ~ +. The effect is invisible to=0A=
applications.=0A=
=0A=
Notes about current patch:=0A=
=0A=
- It does not include ~op=3D operators yet.=0A=
=0A=
- The ~op behaves the same as op on lists, instead of raising=0A=
exceptions.=0A=
=0A=
These should be fixed when the final version of this proposal is=0A=
ready.=0A=
=0A=
- It reserves xor as an infix operator with the semantics=0A=
equivalent to:=0A=
=0A=
def __xor__(a, b):=0A=
if not b: return a=0A=
elif not a: return b=0A=
else: 0=0A=
=0A=
This preserves true value as much as possible, otherwise preserve=0A=
left hand side value if possible.=0A=
=0A=
This is done so that bitwise operators could be regarded as=0A=
elementwise logical operators in the future (see below).=0A=
=0A=
=0A=
Alternatives to adding new operators=0A=
=0A=
The discussions on comp.lang.python and python-dev mailing list=0A=
explored many alternatives. Some of the leading alternatives are=0A=
listed here, using the multiplication operator as an example.=0A=
=0A=
1. Use function mul(a,b).=0A=
=0A=
Advantage:=0A=
- No need for new operators.=0A=
=0A=
Disadvantage: =0A=
- Prefix forms are cumbersome for composite formulas.=0A=
- Unfamiliar to the intended users.=0A=
- Too verbose for the intended users.=0A=
- Unable to use natural precedence rules.=0A=
=0A=
2. Use method call a.mul(b)=0A=
=0A=
Advantage:=0A=
- No need for new operators.=0A=
=0A=
Disadvantage:=0A=
- Asymmetric for both operands.=0A=
- Unfamiliar to the intended users.=0A=
- Too verbose for the intended users.=0A=
- Unable to use natural precedence rules.=0A=
=0A=
3. Use "shadow classes". For matrix class define a shadow array=0A=
class accessible through a method .E, so that for matrices a=0A=
and b, a.E*b would be a matrix object that is=0A=
elementwise_mul(a,b).=0A=
=0A=
Likewise define a shadow matrix class for arrays accessible=0A=
through a method .M so that for arrays a and b, a.M*b would be=0A=
an array that is matrixwise_mul(a,b).=0A=
=0A=
Advantage:=0A=
- No need for new operators.=0A=
- Benefits of infix operators with correct precedence rules.=0A=
- Clean formulas in applications.=0A=
=0A=
Disadvantage:=0A=
- Hard to maintain in current Python because ordinary numbers=0A=
cannot have user defined class methods; i.e. a.E*b will fail=0A=
if a is a pure number.=0A=
- Difficult to implement, as this will interfere with existing=0A=
method calls, like .T for transpose, etc.=0A=
- Runtime overhead of object creation and method lookup.=0A=
- The shadowing class cannot replace a true class, because it=0A=
does not return its own type. So there need to be a M class=0A=
with shadow E class, and an E class with shadow M class.=0A=
- Unnatural to mathematicians.=0A=
=0A=
4. Implement matrixwise and elementwise classes with easy casting=0A=
to the other class. So matrixwise operations for arrays would=0A=
be like a.M*b.M and elementwise operations for matrices would=0A=
be like a.E*b.E. For error detection a.E*b.M would raise=0A=
exceptions.=0A=
=0A=
Advantage:=0A=
- No need for new operators.=0A=
- Similar to infix notation with correct precedence rules.=0A=
=0A=
Disadvantage:=0A=
- Similar difficulty due to lack of user-methods for pure numbers.=0A=
- Runtime overhead of object creation and method lookup.=0A=
- More cluttered formulas=0A=
- Switching of flavor of objects to facilitate operators=0A=
becomes persistent. This introduces long range context=0A=
dependencies in application code that would be extremely hard=0A=
to maintain.=0A=
=0A=
5. Using mini parser to parse formulas written in arbitrary=0A=
extension placed in quoted strings.=0A=
=0A=
Advantage:=0A=
- Pure Python, without new operators=0A=
=0A=
Disadvantage:=0A=
- The actual syntax is within the quoted string, which does not=0A=
resolve the problem itself.=0A=
- Introducing zones of special syntax.=0A=
- Demanding on the mini-parser.=0A=
=0A=
6. Introducing a single operator, such as @, for matrix=0A=
multiplication.=0A=
=0A=
Advantage:=0A=
- Introduces less operators=0A=
=0A=
Disadvantage:=0A=
- The distinctions for operators like + - ** are equally=0A=
important. Their meaning in matrix or array-oriented=0A=
packages would be reversed (see below).=0A=
- The new operator occupies a special character.=0A=
- This does not work well with more general object-element issues.=0A=
=0A=
Among these alternatives, the first and second are used in current=0A=
applications to some extent, but found inadequate. The third is=0A=
the most favorite for applications, but it will incur huge=0A=
implementation complexity. The fourth would make applications=0A=
codes very context-sensitive and hard to maintain. These two=0A=
alternatives also share significant implementational difficulties=0A=
due to current type/class split. The fifth appears to create more=0A=
problems than it would solve. The sixth does not cover the same=0A=
range of applications.=0A=
=0A=
=0A=
Alternative forms of infix operators=0A=
=0A=
Two major forms and several minor variants of new infix operators=0A=
were discussed:=0A=
=0A=
- Bracketed form=0A=
=0A=
(op)=0A=
[op]=0A=
{op}=0A=
<op>=0A=
:op:=0A=
~op~=0A=
%op%=0A=
=0A=
- Meta character form=0A=
=0A=
.op=0A=
@op=0A=
~op=0A=
=0A=
Alternatively the meta character is put after the operator.=0A=
=0A=
- Less consistent variations of these themes. These are=0A=
considered unfavorably. For completeness some are listed here=0A=
=0A=
- Use @/ and /@ for left and right division=0A=
- Use [*] and (*) for outer and inner products=0A=
- Use a single operator @ for multiplication.=0A=
=0A=
- Use __call__ to simulate multiplication.=0A=
a(b) or (a)(b)=0A=
=0A=
Criteria for choosing among the representations include:=0A=
=0A=
- No syntactical ambiguities with existing operators. =0A=
=0A=
- Higher readability in actual formulas. This makes the=0A=
bracketed forms unfavorable. See examples below.=0A=
=0A=
- Visually similar to existing math operators.=0A=
=0A=
- Syntactically simple, without blocking possible future=0A=
extensions.=0A=
=0A=
With these criteria the overall winner in bracket form appear to=0A=
be {op}. A clear winner in the meta character form is ~op.=0A=
Comparing these it appears that ~op is the favorite among them=0A=
all.=0A=
=0A=
Some analysis are as follows:=0A=
=0A=
- The .op form is ambiguous: 1.+a would be different from 1 .+a=0A=
=0A=
- The bracket type operators are most favorable when standing=0A=
alone, but not in formulas, as they interfere with visual=0A=
parsing of parenthesis for precedence and function argument.=0A=
This is so for (op) and [op], and somewhat less so for {op}=0A=
and <op>.=0A=
=0A=
- The <op> form has the potential to be confused with < > and =3D=0A=
=0A=
- The @op is not favored because @ is visually heavy (dense,=0A=
more like a letter): a@+b is more readily read as a@ + b=0A=
than a @+ b.=0A=
=0A=
- For choosing meta-characters: Most of existing ASCII symbols=0A=
have already been used. The only three unused are @ $ ?.=0A=
=0A=
=0A=
Semantics of new operators=0A=
=0A=
There are convincing arguments for using either set of operators=0A=
as objectwise or elementwise. Some of them are listed here:=0A=
=0A=
1. op for element, ~op for object=0A=
=0A=
- Consistent with current multiarray interface of Numeric package=0A=
- Consistent with some other languages=0A=
- Perception that elementwise operations are more natural=0A=
- Perception that elementwise operations are used more frequently=0A=
=0A=
2. op for object, ~op for element=0A=
=0A=
- Consistent with current linear algebra interface of MatPy =
package=0A=
- Consistent with some other languages=0A=
- Perception that objectwise operations are more natural=0A=
- Perception that objectwise operations are used more frequently=0A=
- Consistent with the current behavior of operators on lists=0A=
- Allow ~ to be a general elementwise meta-character in future=0A=
extensions.=0A=
=0A=
It is generally agreed upon that =0A=
=0A=
- there is no absolute reason to favor one or the other=0A=
- it is easy to cast from one representation to another in a=0A=
sizable chunk of code, so the other flavor of operators is=0A=
always minority=0A=
- there are other semantic differences that favor existence of=0A=
array-oriented and matrix-oriented packages, even if their=0A=
operators are unified.=0A=
- whatever the decision is taken, codes using existing=0A=
interfaces should not be broken for a very long time.=0A=
=0A=
Therefore not much is lost, and much flexibility retained, if the=0A=
semantic flavors of these two sets of operators are not dictated=0A=
by the core language. The application packages are responsible=0A=
for making the most suitable choice. This is already the case for=0A=
NumPy and MatPy which use opposite semantics. Adding new=0A=
operators will not break this. See also observation after=0A=
subsection 2 in the Examples below.=0A=
=0A=
The issue of numerical precision was raised, but if the semantics=0A=
is left to the applications, the actual precisions should also go=0A=
there.=0A=
=0A=
=0A=
Examples=0A=
=0A=
Following are examples of the actual formulas that will appear=0A=
using various operators or other representations described above.=0A=
=0A=
1. The matrix inversion formula:=0A=
=0A=
- Using op for object and ~op for element:=0A=
=0A=
b =3D a.I - a.I * u / (c.I + v/a*u) * v / a=0A=
=0A=
b =3D a.I - a.I * u * (c.I + v*a.I*u).I * v * a.I=0A=
=0A=
- Using op for element and ~op for object:=0A=
=0A=
b =3D a.I @- a.I @* u @/ (c.I @+ v@/a@*u) @* v @/ a=0A=
=0A=
b =3D a.I ~- a.I ~* u ~/ (c.I ~+ v~/a~*u) ~* v ~/ a=0A=
=0A=
b =3D a.I (-) a.I (*) u (/) (c.I (+) v(/)a(*)u) (*) v (/) a=0A=
=0A=
b =3D a.I [-] a.I [*] u [/] (c.I [+] v[/]a[*]u) [*] v [/] a=0A=
=0A=
b =3D a.I <-> a.I <*> u </> (c.I <+> v</>a<*>u) <*> v </> a=0A=
=0A=
b =3D a.I {-} a.I {*} u {/} (c.I {+} v{/}a{*}u) {*} v {/} a=0A=
=0A=
Observation: For linear algebra using op for object is preferable.=0A=
=0A=
Observation: The ~op type operators look better than (op) type=0A=
in complicated formulas.=0A=
=0A=
- using named operators=0A=
=0A=
b =3D a.I @sub a.I @mul u @div (c.I @add v @div a @mul u) @mul =
v @div a=0A=
=0A=
b =3D a.I ~sub a.I ~mul u ~div (c.I ~add v ~div a ~mul u) ~mul =
v ~div a=0A=
=0A=
Observation: Named operators are not suitable for math formulas.=0A=
=0A=
2. Plotting a 3d graph=0A=
=0A=
- Using op for object and ~op for element:=0A=
=0A=
z =3D sin(x~**2 ~+ y~**2); plot(x,y,z)=0A=
=0A=
- Using op for element and ~op for object:=0A=
=0A=
z =3D sin(x**2 + y**2); plot(x,y,z)=0A=
=0A=
Observation: Elementwise operations with broadcasting allows=0A=
much more efficient implementation than MatLab.=0A=
=0A=
Observation: It is useful to have two related classes with the=0A=
semantics of op and ~op swapped. Using these the ~op=0A=
operators would only need to appear in chunks of code where=0A=
the other flavor dominates, while maintaining consistent=0A=
semantics of the code.=0A=
=0A=
3. Using + and - with automatic broadcasting=0A=
=0A=
a =3D b - c; d =3D a.T*a=0A=
=0A=
Observation: This would silently produce hard-to-trace bugs if=0A=
one of b or c is row vector while the other is column vector.=0A=
=0A=
=0A=
Miscellaneous issues:=0A=
=0A=
- Need for the ~+ ~- operators. The objectwise + - are important=0A=
because they provide important sanity checks as per linear=0A=
algebra. The elementwise + - are important because they allow=0A=
broadcasting that are very efficient in applications.=0A=
=0A=
- Left division (solve). For matrix, a*x is not necessarily equal=0A=
to x*a. The solution of a*x=3D=3Db, denoted x=3Dsolve(a,b), is=0A=
therefore different from the solution of x*a=3D=3Db, denoted=0A=
x=3Ddiv(b,a). There are discussions about finding a new symbol=0A=
for solve. [Background: MatLab use b/a for div(b,a) and a\b for=0A=
solve(a,b).]=0A=
=0A=
It is recognized that Python provides a better solution without=0A=
requiring a new symbol: the inverse method .I can be made to be=0A=
delayed so that a.I*b and b*a.I are equivalent to Mat lab's a\b=0A=
and b/a. The implementation is quite simple and the resulting=0A=
application code clean.=0A=
=0A=
- Power operator. Python's use of a**b as pow(a,b) has two=0A=
perceived disadvantages:=0A=
=0A=
- Most mathematicians are more familiar with a^b for this purpose.=0A=
- It results in long augmented assignment operator ~**=3D.=0A=
=0A=
However, this issue is distinct from the main issue here.=0A=
=0A=
- Additional multiplication operators. Several forms of=0A=
multiplications are used in (multi-)linear algebra. Most can be=0A=
seen as variations of multiplication in linear algebra sense=0A=
(such as Kronecker product). But two forms appear to be more=0A=
fundamental: outer product and inner product. However, their=0A=
specification includes indices, which can be either=0A=
=0A=
- associated with the operator, or=0A=
- associated with the objects.=0A=
=0A=
The latter (the Einstein notation) is used extensively on paper,=0A=
and is also the easier one to implement. By implementing a=0A=
tensor-with-indices class, a general form of multiplication=0A=
would cover both outer and inner products, and specialize to=0A=
linear algebra multiplication as well. The index rule can be=0A=
defined as class methods, like,=0A=
=0A=
a =3D b.i(1,2,-1,-2) * c.i(4,-2,3,-1) # a_ijkl =3D b_ijmn =
c_lnkm=0A=
=0A=
Therefore one objectwise multiplication is sufficient.=0A=
=0A=
- Bitwise operators. =0A=
=0A=
- The proposed new math operators use the symbol ~ that is=0A=
"bitwise not" operator. This poses no compatibility problem=0A=
but somewhat complicates implementation.=0A=
=0A=
- The symbol ^ might be better used for pow than bitwise xor.=0A=
But this depends on the future of bitwise operators. It does=0A=
not immediately impact on the proposed math operator.=0A=
=0A=
- The symbol | was suggested to be used for matrix solve. But=0A=
the new solution of using delayed .I is better in several=0A=
ways.=0A=
=0A=
- The current proposal fits in a larger and more general=0A=
extension that will remove the need for special bitwise=0A=
operators. (See elementization below.)=0A=
=0A=
- Alternative to special operator names used in definition,=0A=
=0A=
def "+"(a, b) in place of def __add__(a, b)=0A=
=0A=
This appears to require greater syntactical change, and would=0A=
only be useful when arbitrary additional operators are allowed.=0A=
=0A=
=0A=
Impact on general elementization=0A=
=0A=
The distinction between objectwise and elementwise operations are=0A=
meaningful in other contexts as well, where an object can be=0A=
conceptually regarded as a collection of elements. It is=0A=
important that the current proposal does not preclude possible=0A=
future extensions.=0A=
=0A=
One general future extension is to use ~ as a meta operator to=0A=
"elementize" a given operator. Several examples are listed here:=0A=
=0A=
1. Bitwise operators. Currently Python assigns six operators to=0A=
bitwise operations: and (&), or (|), xor (^), complement (~),=0A=
left shift (<<) and right shift (>>), with their own precedence=0A=
levels.=0A=
=0A=
Among them, the & | ^ ~ operators can be regarded as=0A=
elementwise versions of lattice operators applied to integers=0A=
regarded as bit strings.=0A=
=0A=
5 and 6 # 6=0A=
5 or 6 # 5=0A=
=0A=
5 ~and 6 # 4=0A=
5 ~or 6 # 7=0A=
=0A=
These can be regarded as general elementwise lattice operators,=0A=
not restricted to bits in integers.=0A=
=0A=
In order to have named operators for xor ~xor, it is necessary=0A=
to make xor a reserved word.=0A=
=0A=
2. List arithmetics. =0A=
=0A=
[1, 2] + [3, 4] # [1, 2, 3, 4]=0A=
[1, 2] ~+ [3, 4] # [4, 6]=0A=
=0A=
['a', 'b'] * 2 # ['a', 'b', 'a', 'b']=0A=
'ab' * 2 # 'abab'=0A=
=0A=
['a', 'b'] ~* 2 # ['aa', 'bb']=0A=
[1, 2] ~* 2 # [2, 4]=0A=
=0A=
It is also consistent to Cartesian product=0A=
=0A=
[1,2]*[3,4] # [(1,3),(1,4),(2,3),(2,4)]=0A=
=0A=
3. List comprehension.=0A=
=0A=
a =3D [1, 2]; b =3D [3, 4]=0A=
~f(a,b) # [f(x,y) for x, y in zip(a,b)]=0A=
~f(a*b) # [f(x,y) for x in a for y in b]=0A=
a ~+ b # [x + y for x, y in zip(a,b)]=0A=
=0A=
4. Tuple generation (the zip function in Python 2.0)=0A=
=0A=
[1, 2, 3], [4, 5, 6] # ([1,2, 3], [4, 5, 6])=0A=
[1, 2, 3]~,[4, 5, 6] # [(1,4), (2, 5), (3,6)]=0A=
=0A=
5. Using ~ as generic elementwise meta-character to replace map=0A=
=0A=
~f(a, b) # map(f, a, b)=0A=
~~f(a, b) # map(lambda *x:map(f, *x), a, b)=0A=
=0A=
More generally,=0A=
=0A=
def ~f(*x): return map(f, *x)=0A=
def ~~f(*x): return map(~f, *x)=0A=
...=0A=
=0A=
6. Elementwise format operator (with broadcasting)=0A=
=0A=
a =3D [1,2,3,4,5]=0A=
print ["%5d "] ~% a =0A=
a =3D [[1,2],[3,4]]=0A=
print ["%5d "] ~~% a=0A=
=0A=
7. Rich comparison=0A=
=0A=
[1, 2, 3] ~< [3, 2, 1] # [1, 0, 0]=0A=
[1, 2, 3] ~=3D=3D [3, 2, 1] # [0, 1, 0]=0A=
=0A=
8. Rich indexing=0A=
=0A=
[a, b, c, d] ~[2, 3, 1] # [c, d, b]=0A=
=0A=
9. Tuple flattening=0A=
=0A=
a =3D (1,2); b =3D (3,4)=0A=
f(~a, ~b) # f(1,2,3,4) =0A=
=0A=
10. Copy operator=0A=
=0A=
a ~=3D b # a =3D b.copy()=0A=
=0A=
There can be specific levels of deep copy=0A=
=0A=
a ~~=3D b # a =3D b.copy(2)=0A=
=0A=
Notes:=0A=
=0A=
1. There are probably many other similar situations. This general=0A=
approach seems well suited for most of them, in place of=0A=
several separated extensions for each of them (parallel and=0A=
cross iteration, list comprehension, rich comparison, etc).=0A=
=0A=
2. The semantics of "elementwise" depends on applications. For=0A=
example, an element of matrix is two levels down from the=0A=
list-of-list point of view. This requires more fundamental=0A=
change than the current proposal. In any case, the current=0A=
proposal will not negatively impact on future possibilities of=0A=
this nature.=0A=
=0A=
Note that this section describes a type of future extensions that=0A=
is consistent with current proposal, but may present additional=0A=
compatibility or other problems. They are not tied to the current=0A=
proposal.=0A=
=0A=
=0A=
Impact on named operators=0A=
=0A=
The discussions made it generally clear that infix operators is a=0A=
scarce resource in Python, not only in numerical computation, but=0A=
in other fields as well. Several proposals and ideas were put=0A=
forward that would allow infix operators be introduced in ways=0A=
similar to named functions. We show here that the current=0A=
extension does not negatively impact on future extensions in this=0A=
regard.=0A=
=0A=
1. Named infix operators.=0A=
=0A=
Choose a meta character, say @, so that for any identifier=0A=
"opname", the combination "@opname" would be a binary infix=0A=
operator, and=0A=
=0A=
a @opname b =3D=3D opname(a,b)=0A=
=0A=
Other representations mentioned include .name ~name~ :name:=0A=
(.name) %name% and similar variations. The pure bracket based=0A=
operators cannot be used this way.=0A=
=0A=
This requires a change in the parser to recognize @opname, and=0A=
parse it into the same structure as a function call. The=0A=
precedence of all these operators would have to be fixed at=0A=
one level, so the implementation would be different from=0A=
additional math operators which keep the precedence of=0A=
existing math operators.=0A=
=0A=
The current proposed extension do not limit possible future=0A=
extensions of such form in any way.=0A=
=0A=
2. More general symbolic operators.=0A=
=0A=
One additional form of future extension is to use meta=0A=
character and operator symbols (symbols that cannot be used in=0A=
syntactical structures other than operators). Suppose @ is=0A=
the meta character. Then=0A=
=0A=
a + b, a @+ b, a @@+ b, a @+- b=0A=
=0A=
would all be operators with a hierarchy of precedence, defined by=0A=
=0A=
def "+"(a, b)=0A=
def "@+"(a, b)=0A=
def "@@+"(a, b)=0A=
def "@+-"(a, b)=0A=
=0A=
One advantage compared with named operators is greater=0A=
flexibility for precedences based on either the meta character=0A=
or the ordinary operator symbols. This also allows operator=0A=
composition. The disadvantage is that they are more like=0A=
"line noise". In any case the current proposal does not=0A=
impact its future possibility.=0A=
=0A=
These kinds of future extensions may not be necessary when=0A=
Unicode becomes generally available.=0A=
=0A=
Note that this section discusses compatibility of the proposed=0A=
extension with possible future extensions. The desirability=0A=
or compatibility of these other extensions themselves are=0A=
specifically not considered here.=0A=
=0A=
=0A=
Credits and archives=0A=
=0A=
The discussions mostly happened in July to August of 2000 on news=0A=
group comp.lang.python and the mailing list python-dev. There are=0A=
altogether several hundred postings, most can be retrieved from=0A=
these two pages (and searching word "operator"):=0A=
=0A=
http://www.python.org/pipermail/python-list/2000-July/=0A=
http://www.python.org/pipermail/python-list/2000-August/=0A=
=0A=
The names of contributers are too numerous to mention here,=0A=
suffice to say that a large proportion of ideas discussed here are=0A=
not our own.=0A=
=0A=
Several key postings (from our point of view) that may help to=0A=
navigate the discussions include:=0A=
=0A=
http://www.python.org/pipermail/python-list/2000-July/108893.html=0A=
http://www.python.org/pipermail/python-list/2000-July/108777.html=0A=
http://www.python.org/pipermail/python-list/2000-July/108848.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109237.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109250.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109310.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109448.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109491.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109537.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109607.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109709.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109804.html=0A=
http://www.python.org/pipermail/python-list/2000-July/109857.html=0A=
http://www.python.org/pipermail/python-list/2000-July/110061.html=0A=
http://www.python.org/pipermail/python-list/2000-July/110208.html=0A=
=
http://www.python.org/pipermail/python-list/2000-August/111427.html=0A=
=
http://www.python.org/pipermail/python-list/2000-August/111558.html=0A=
=
http://www.python.org/pipermail/python-list/2000-August/112551.html=0A=
=
http://www.python.org/pipermail/python-list/2000-August/112606.html=0A=
=
http://www.python.org/pipermail/python-list/2000-August/112758.html=0A=
=0A=
http://www.python.org/pipermail/python-dev/2000-July/013243.html=0A=
http://www.python.org/pipermail/python-dev/2000-July/013364.html=0A=
=
http://www.python.org/pipermail/python-dev/2000-August/014940.html=0A=
=0A=
These are earlier drafts of this PEP:=0A=
=0A=
=
http://www.python.org/pipermail/python-list/2000-August/111785.html=0A=
=
http://www.python.org/pipermail/python-list/2000-August/112529.html=0A=
=
http://www.python.org/pipermail/python-dev/2000-August/014906.html=0A=
=0A=
There is an alternative PEP (with official PEP number 211) by Greg=0A=
Wilson, titled "Adding New Linear Algebra Operators to Python".=0A=
=0A=
Its first (and current) version is at:=0A=
=0A=
=
http://www.python.org/pipermail/python-dev/2000-August/014876.html=0A=
http://python.sourceforge.net/peps/pep-0211.html=0A=
=0A=
=0A=
Additional References=0A=
=0A=
[1] http://MatPy.sourceforge.net/Misc/index.html=0A=
=0A=
=0A=
=0C=0A=
Local Variables:=0A=
mode: indented-text=0A=
indent-tabs-mode: nil=0A=
End:=0A=
------=_NextPart_000_002A_01C066B6.32690BC0--