Python History -------------- This file contains the release messages for previous Python releases. As you read on you go back to the dark ages of Python's history. (Note: news about 2.5c2 and later 2.5 releases is in the Misc/NEWS file of the release25-maint branch.) ====================================================================== What's New in Python 2.5 release candidate 1? ============================================= *Release date: 17-AUG-2006* Core and builtins ----------------- - Unicode objects will no longer raise an exception when being compared equal or unequal to a string and a UnicodeDecodeError exception occurs, e.g. as result of a decoding failure. Instead, the equal (==) and unequal (!=) comparison operators will now issue a UnicodeWarning and interpret the two objects as unequal. The UnicodeWarning can be filtered as desired using the warning framework, e.g. silenced completely, turned into an exception, logged, etc. Note that compare operators other than equal and unequal will still raise UnicodeDecodeError exceptions as they've always done. - Fix segfault when doing string formatting on subclasses of long. - Fix bug related to __len__ functions using values > 2**32 on 64-bit machines with new-style classes. - Fix bug related to __len__ functions returning negative values with classic classes. - Patch #1538606, Fix __index__() clipping. There were some problems discovered with the API and how integers that didn't fit into Py_ssize_t were handled. This patch attempts to provide enough alternatives to effectively use __index__. - Bug #1536021: __hash__ may now return long int; the final hash value is obtained by invoking hash on the long int. - Bug #1536786: buffer comparison could emit a RuntimeWarning. - Bug #1535165: fixed a segfault in input() and raw_input() when sys.stdin is closed. - On Windows, the PyErr_Warn function is now exported from the Python dll again. - Bug #1191458: tracing over for loops now produces a line event on each iteration. Fixing this problem required changing the .pyc magic number. This means that .pyc files generated before 2.5c1 will be regenerated. - Bug #1333982: string/number constants were inappropriately stored in the byte code and co_consts even if they were not used, ie immediately popped off the stack. - Fixed a reference-counting problem in property(). Library ------- - Fix a bug in the ``compiler`` package that caused invalid code to be generated for generator expressions. - The distutils version has been changed to 2.5.0. The change to keep it programmatically in sync with the Python version running the code (introduced in 2.5b3) has been reverted. It will continue to be maintained manually as static string literal. - If the Python part of a ctypes callback function returns None, and this cannot be converted to the required C type, an exception is printed with PyErr_WriteUnraisable. Before this change, the C callback returned arbitrary values to the calling code. - The __repr__ method of a NULL ctypes.py_object() no longer raises an exception. - uuid.UUID now has a bytes_le attribute. This returns the UUID in little-endian byte order for Windows. In addition, uuid.py gained some workarounds for clocks with low resolution, to stop the code yielding duplicate UUIDs. - Patch #1540892: site.py Quitter() class attempts to close sys.stdin before raising SystemExit, allowing IDLE to honor quit() and exit(). - Bug #1224621: make tabnanny recognize IndentationErrors raised by tokenize. - Patch #1536071: trace.py should now find the full module name of a file correctly even on Windows. - logging's atexit hook now runs even if the rest of the module has already been cleaned up. - Bug #1112549, fix DoS attack on cgi.FieldStorage. - Bug #1531405, format_exception no longer raises an exception if str(exception) raised an exception. - Fix a bug in the ``compiler`` package that caused invalid code to be generated for nested functions. Extension Modules ----------------- - Patch #1511317: don't crash on invalid hostname (alias) info. - Patch #1535500: fix segfault in BZ2File.writelines and make sure it raises the correct exceptions. - Patch # 1536908: enable building ctypes on OpenBSD/AMD64. The '-no-stack-protector' compiler flag for OpenBSD has been removed. - Patch #1532975 was applied, which fixes Bug #1533481: ctypes now uses the _as_parameter_ attribute when objects are passed to foreign function calls. The ctypes version number was changed to 1.0.1. - Bug #1530559, struct.pack raises TypeError where it used to convert. Passing float arguments to struct.pack when integers are expected now triggers a DeprecationWarning. Tests ----- - test_socketserver should now work on cygwin and not fail sporadically on other platforms. - test_mailbox should now work on cygwin versions 2006-08-10 and later. - Bug #1535182: really test the xreadlines() method of bz2 objects. - test_threading now skips testing alternate thread stack sizes on platforms that don't support changing thread stack size. Documentation ------------- - Patch #1534922: unittest docs were corrected and enhanced. Build ----- - Bug #1535502, build _hashlib on Windows, and use masm assembler code in OpenSSL. - Bug #1534738, win32 debug version of _msi should be _msi_d.pyd. - Bug #1530448, ctypes build failure on Solaris 10 was fixed. C API ----- - New API for Unicode rich comparisons: PyUnicode_RichCompare() - Bug #1069160. Internal correctness changes were made to ``PyThreadState_SetAsyncExc()``. A test case was added, and the documentation was changed to state that the return value is always 1 (normal) or 0 (if the specified thread wasn't found). What's New in Python 2.5 beta 3? ================================ *Release date: 03-AUG-2006* Core and builtins ----------------- - _PyWeakref_GetWeakrefCount() now returns a Py_ssize_t; it previously returned a long (see PEP 353). - Bug #1515471: string.replace() accepts character buffers again. - Add PyErr_WarnEx() so C code can pass the stacklevel to warnings.warn(). This provides the proper warning for struct.pack(). PyErr_Warn() is now deprecated in favor of PyErr_WarnEx(). - Patch #1531113: Fix augmented assignment with yield expressions. Also fix a SystemError when trying to assign to yield expressions. - Bug #1529871: The speed enhancement patch #921466 broke Python's compliance with PEP 302. This was fixed by adding an ``imp.NullImporter`` type that is used in ``sys.path_importer_cache`` to cache non-directory paths and avoid excessive filesystem operations during imports. - Bug #1521947: When checking for overflow, ``PyOS_strtol()`` used some operations on signed longs that are formally undefined by C. Unfortunately, at least one compiler now cares about that, so complicated the code to make that compiler happy again. - Bug #1524310: Properly report errors from FindNextFile in os.listdir. - Patch #1232023: Stop including current directory in search path on Windows. - Fix some potential crashes found with failmalloc. - Fix warnings reported by Klocwork's static analysis tool. - Bug #1512814, Fix incorrect lineno's when code within a function had more than 255 blank lines. - Patch #1521179: Python now accepts the standard options ``--help`` and ``--version`` as well as ``/?`` on Windows. - Bug #1520864: unpacking singleton tuples in a 'for' loop (for x, in) works again. Fixing this problem required changing the .pyc magic number. This means that .pyc files generated before 2.5b3 will be regenerated. - Bug #1524317: Compiling Python ``--without-threads`` failed. The Python core compiles again, and, in a build without threads, the new ``sys._current_frames()`` returns a dictionary with one entry, mapping the faux "thread id" 0 to the current frame. - Bug #1525447: build on MacOS X on a case-sensitive filesystem. Library ------- - Fix #1693149. Now you can pass several modules separated by comma to trace.py in the same --ignore-module option. - Correction of patch #1455898: In the mbcs decoder, set final=False for stream decoder, but final=True for the decode function. - os.urandom no longer masks unrelated exceptions like SystemExit or KeyboardInterrupt. - Bug #1525866: Don't copy directory stat times in shutil.copytree on Windows - Bug #1002398: The documentation for os.path.sameopenfile now correctly refers to file descriptors, not file objects. - The renaming of the xml package to xmlcore, and the import hackery done to make it appear at both names, has been removed. Bug #1511497, #1513611, and probably others. - Bug #1441397: The compiler module now recognizes module and function docstrings correctly as it did in Python 2.4. - Bug #1529297: The rewrite of doctest for Python 2.4 unintentionally lost that tests are sorted by name before being run. This rarely matters for well-written tests, but can create baffling symptoms if side effects from one test to the next affect outcomes. ``DocTestFinder`` has been changed to sort the list of tests it returns. - The distutils version has been changed to 2.5.0, and is now kept in sync with sys.version_info[:3]. - Bug #978833: Really close underlying socket in _socketobject.close. - Bug #1459963: urllib and urllib2 now normalize HTTP header names with title(). - Patch #1525766: In pkgutil.walk_packages, correctly pass the onerror callback to recursive calls and call it with the failing package name. - Bug #1525817: Don't truncate short lines in IDLE's tool tips. - Patch #1515343: Fix printing of deprecated string exceptions with a value in the traceback module. - Resync optparse with Optik 1.5.3: minor tweaks for/to tests. - Patch #1524429: Use repr() instead of backticks in Tkinter again. - Bug #1520914: Change time.strftime() to accept a zero for any position in its argument tuple. For arguments where zero is illegal, the value is forced to the minimum value that is correct. This is to support an undocumented but common way people used to fill in inconsequential information in the time tuple pre-2.4. - Patch #1220874: Update the binhex module for Mach-O. - The email package has improved RFC 2231 support, specifically for recognizing the difference between encoded (name*0*=) and non-encoded (name*0=) parameter continuations. This may change the types of values returned from email.message.Message.get_param() and friends. Specifically in some cases where non-encoded continuations were used, get_param() used to return a 3-tuple of (None, None, string) whereas now it will just return the string (since non-encoded continuations don't have charset and language parts). Also, whereas % values were decoded in all parameter continuations, they are now only decoded in encoded parameter parts. - Bug #1517990: IDLE keybindings on MacOS X now work correctly - Bug #1517996: IDLE now longer shows the default Tk menu when a path browser, class browser or debugger is the frontmost window on MacOS X - Patch #1520294: Support for getset and member descriptors in types.py, inspect.py, and pydoc.py. Specifically, this allows for querying the type of an object against these built-in types and more importantly, for getting their docstrings printed in the interactive interpreter's help() function. Extension Modules ----------------- - Patch #1519025 and bug #926423: If a KeyboardInterrupt occurs during a socket operation on a socket with a timeout, the exception will be caught correctly. Previously, the exception was not caught. - Patch #1529514: The _ctypes extension is now compiled on more openbsd target platforms. - The ``__reduce__()`` method of the new ``collections.defaultdict`` had a memory leak, affecting pickles and deep copies. - Bug #1471938: Fix curses module build problem on Solaris 8; patch by Paul Eggert. - Patch #1448199: Release interpreter lock in _winreg.ConnectRegistry. - Patch #1521817: Index range checking on ctypes arrays containing exactly one element enabled again. This allows iterating over these arrays, without the need to check the array size before. - Bug #1521375: When the code in ctypes.util.find_library was run with root privileges, it could overwrite or delete /dev/null in certain cases; this is now fixed. - Bug #1467450: On Mac OS X 10.3, RTLD_GLOBAL is now used as the default mode for loading shared libraries in ctypes. - Because of a misspelled preprocessor symbol, ctypes was always compiled without thread support; this is now fixed. - pybsddb Bug #1527939: bsddb module DBEnv dbremove and dbrename methods now allow their database parameter to be None as the sleepycat API allows. - Bug #1526460: Fix socketmodule compile on NetBSD as it has a different bluetooth API compared with Linux and FreeBSD. Tests ----- - Bug #1501330: Change test_ossaudiodev to be much more tolerant in terms of how long the test file should take to play. Now accepts taking 2.93 secs (exact time) +/- 10% instead of the hard-coded 3.1 sec. - Patch #1529686: The standard tests ``test_defaultdict``, ``test_iterlen``, ``test_uuid`` and ``test_email_codecs`` didn't actually run any tests when run via ``regrtest.py``. Now they do. Build ----- - Bug #1439538: Drop usage of test -e in configure as it is not portable. Mac --- - PythonLauncher now works correctly when the path to the script contains characters that are treated specially by the shell (such as quotes). - Bug #1527397: PythonLauncher now launches scripts with the working directory set to the directory that contains the script instead of the user home directory. That latter was an implementation accident and not what users expect. What's New in Python 2.5 beta 2? ================================ *Release date: 11-JUL-2006* Core and builtins ----------------- - Bug #1441486: The literal representation of -(sys.maxint - 1) again evaluates to a int object, not a long. - Bug #1501934: The scope of global variables that are locally assigned using augmented assignment is now correctly determined. - Bug #927248: Recursive method-wrapper objects can now safely be released. - Bug #1417699: Reject locale-specific decimal point in float() and atof(). - Bug #1511381: codec_getstreamcodec() in codec.c is corrected to omit a default "error" argument for NULL pointer. This allows the parser to take a codec from cjkcodecs again. - Bug #1519018: 'as' is now validated properly in import statements. - On 64 bit systems, int literals that use less than 64 bits are now ints rather than longs. - Bug #1512814, Fix incorrect lineno's when code at module scope started after line 256. - New function ``sys._current_frames()`` returns a dict mapping thread id to topmost thread stack frame. This is for expert use, and is especially useful for debugging application deadlocks. The functionality was previously available in Fazal Majid's ``threadframe`` extension module, but it wasn't possible to do this in a wholly threadsafe way from an extension. Library ------- - Bug #1257728: Mention Cygwin in distutils error message about a missing VS 2003. - Patch #1519566: Update turtle demo, make begin_fill idempotent. - Bug #1508010: msvccompiler now requires the DISTUTILS_USE_SDK environment variable to be set in order to the SDK environment for finding the compiler, include files, etc. - Bug #1515998: Properly generate logical ids for files in bdist_msi. - warnings.py now ignores ImportWarning by default - string.Template() now correctly handles tuple-values. Previously, multi-value tuples would raise an exception and single-value tuples would be treated as the value they contain, instead. - Bug #822974: Honor timeout in telnetlib.{expect,read_until} even if some data are received. - Bug #1267547: Put proper recursive setup.py call into the spec file generated by bdist_rpm. - Bug #1514693: Update turtle's heading when switching between degrees and radians. - Reimplement turtle.circle using a polyline, to allow correct filling of arcs. - Bug #1514703: Only setup canvas window in turtle when the canvas is created. - Bug #1513223: .close() of a _socketobj now releases the underlying socket again, which then gets closed as it becomes unreferenced. - Bug #1504333: Make sgmllib support angle brackets in quoted attribute values. - Bug #853506: Fix IPv6 address parsing in unquoted attributes in sgmllib ('[' and ']' were not accepted). - Fix a bug in the turtle module's end_fill function. - Bug #1510580: The 'warnings' module improperly required that a Warning category be either a types.ClassType and a subclass of Warning. The proper check is just that it is a subclass with Warning as the documentation states. - The compiler module now correctly compiles the new try-except-finally statement (bug #1509132). - The wsgiref package is now installed properly on Unix. - A bug was fixed in logging.config.fileConfig() which caused a crash on shutdown when fileConfig() was called multiple times. - The sqlite3 module did cut off data from the SQLite database at the first null character before sending it to a custom converter. This has been fixed now. Extension Modules ----------------- - #1494314: Fix a regression with high-numbered sockets in 2.4.3. This means that select() on sockets > FD_SETSIZE (typically 1024) work again. The patch makes sockets use poll() internally where available. - Assigning None to pointer type fields in ctypes structures possible overwrote the wrong fields, this is fixed now. - Fixed a segfault in _ctypes when ctypes.wintypes were imported on non-Windows platforms. - Bug #1518190: The ctypes.c_void_p constructor now accepts any integer or long, without range checking. - Patch #1517790: It is now possible to use custom objects in the ctypes foreign function argtypes sequence as long as they provide a from_param method, no longer is it required that the object is a ctypes type. - The '_ctypes' extension module now works when Python is configured with the --without-threads option. - Bug #1513646: os.access on Windows now correctly determines write access, again. - Bug #1512695: cPickle.loads could crash if it was interrupted with a KeyboardInterrupt. - Bug #1296433: parsing XML with a non-default encoding and a CharacterDataHandler could crash the interpreter in pyexpat. - Patch #1516912: improve Modules support for OpenVMS. Build ----- - Automate Windows build process for the Win64 SSL module. - 'configure' now detects the zlib library the same way as distutils. Previously, the slight difference could cause compilation errors of the 'zlib' module on systems with more than one version of zlib. - The MSI compileall step was fixed to also support a TARGETDIR with spaces in it. - Bug #1517388: sqlite3.dll is now installed on Windows independent of Tcl/Tk. - Bug #1513032: 'make install' failed on FreeBSD 5.3 due to lib-old trying to be installed even though it's empty. Tests ----- - Call os.waitpid() at the end of tests that spawn child processes in order to minimize resources (zombies). Documentation ------------- - Cover ImportWarning, PendingDeprecationWarning and simplefilter() in the documentation for the warnings module. - Patch #1509163: MS Toolkit Compiler no longer available. - Patch #1504046: Add documentation for xml.etree. What's New in Python 2.5 beta 1? ================================ *Release date: 20-JUN-2006* Core and builtins ----------------- - Patch #1507676: Error messages returned by invalid abstract object operations (such as iterating over an integer) have been improved and now include the type of the offending object to help with debugging. - Bug #992017: A classic class that defined a __coerce__() method that returned its arguments swapped would infinitely recurse and segfault the interpreter. - Fix the socket tests so they can be run concurrently. - Removed 5 integers from C frame objects (PyFrameObject). f_nlocals, f_ncells, f_nfreevars, f_stack_size, f_restricted. - Bug #532646: object.__call__() will continue looking for the __call__ attribute on objects until one without one is found. This leads to recursion when you take a class and set its __call__ attribute to an instance of the class. Originally fixed for classic classes, but this fix is for new-style. Removes the infinite_rec_3 crasher. - The string and unicode methods startswith() and endswith() now accept a tuple of prefixes/suffixes to look for. Implements RFE #1491485. - Buffer objects, at the C level, never used the char buffer implementation even when the char buffer for the wrapped object was explicitly requested (originally returned the read or write buffer). Now a TypeError is raised if the char buffer is not present but is requested. - Patch #1346214: Statements like "if 0: suite" are now again optimized away like they were in Python 2.4. - Builtin exceptions are now full-blown new-style classes instead of instances pretending to be classes, which speeds up exception handling by about 80% in comparison to 2.5a2. - Patch #1494554: Update unicodedata.numeric and unicode.isnumeric to Unicode 4.1. - Patch #921466: sys.path_importer_cache is now used to cache valid and invalid file paths for the built-in import machinery which leads to fewer open calls on startup. - Patch #1442927: ``long(str, base)`` is now up to 6x faster for non-power- of-2 bases. The largest speedup is for inputs with about 1000 decimal digits. Conversion from non-power-of-2 bases remains quadratic-time in the number of input digits (it was and remains linear-time for bases 2, 4, 8, 16 and 32). - Bug #1334662: ``int(string, base)`` could deliver a wrong answer when ``base`` was not 2, 4, 8, 10, 16 or 32, and ``string`` represented an integer close to ``sys.maxint``. This was repaired by patch #1335972, which also gives a nice speedup. - Patch #1337051: reduced size of frame objects. - PyErr_NewException now accepts a tuple of base classes as its "base" parameter. - Patch #876206: function call speedup by retaining allocated frame objects. - Bug #1462152: file() now checks more thoroughly for invalid mode strings and removes a possible "U" before passing the mode to the C library function. - Patch #1488312, Fix memory alignment problem on SPARC in unicode - Bug #1487966: Fix SystemError with conditional expression in assignment - WindowsError now has two error code attributes: errno, which carries the error values from errno.h, and winerror, which carries the error values from winerror.h. Previous versions put the winerror.h values (from GetLastError()) into the errno attribute. - Patch #1475845: Raise IndentationError for unexpected indent. - Patch #1479181: split open() and file() from being aliases for each other. - Patch #1497053 & bug #1275608: Exceptions occurring in ``__eq__()`` methods were always silently ignored by dictionaries when comparing keys. They are now passed through (except when using the C API function ``PyDict_GetItem()``, whose semantics did not change). - Bug #1456209: In some obscure cases it was possible for a class with a custom ``__eq__()`` method to confuse dict internals when class instances were used as a dict's keys and the ``__eq__()`` method mutated the dict. No, you don't have any code that did this ;-) Extension Modules ----------------- - Bug #1295808: expat symbols should be namespaced in pyexpat - Patch #1462338: Upgrade pyexpat to expat 2.0.0 - Change binascii.hexlify to accept a read-only buffer instead of only a char buffer and actually follow its documentation. - Fixed a potentially invalid memory access of CJKCodecs' shift-jis decoder. - Patch #1478788 (modified version): The functional extension module has been renamed to _functools and a functools Python wrapper module added. This provides a home for additional function related utilities that are not specifically about functional programming. See PEP 309. - Patch #1493701: performance enhancements for struct module. - Patch #1490224: time.altzone is now set correctly on Cygwin. - Patch #1435422: zlib's compress and decompress objects now have a copy() method. - Patch #1454481: thread stack size is now tunable at runtime for thread enabled builds on Windows and systems with Posix threads support. - On Win32, os.listdir now supports arbitrarily-long Unicode path names (up to the system limit of 32K characters). - Use Win32 API to implement os.{access,chdir,chmod,mkdir,remove,rename,rmdir,utime}. As a result, these functions now raise WindowsError instead of OSError. - ``time.clock()`` on Win64 should use the high-performance Windows ``QueryPerformanceCounter()`` now (as was already the case on 32-bit Windows platforms). - Calling Tk_Init twice is refused if the first call failed as that may deadlock. - bsddb: added the DB_ARCH_REMOVE flag and fixed db.DBEnv.log_archive() to accept it without potentially using an uninitialized pointer. - bsddb: added support for the DBEnv.log_stat() and DBEnv.lsn_reset() methods assuming BerkeleyDB >= 4.0 and 4.4 respectively. [pybsddb project SF patch numbers 1494885 and 1494902] - bsddb: added an interface for the BerkeleyDB >= 4.3 DBSequence class. [pybsddb project SF patch number 1466734] - bsddb: fix DBCursor.pget() bug with keyword argument names when no data parameter is supplied. [SF pybsddb bug #1477863] - bsddb: the __len__ method of a DB object has been fixed to return correct results. It could previously incorrectly return 0 in some cases. Fixes SF bug 1493322 (pybsddb bug 1184012). - bsddb: the bsddb.dbtables Modify method now raises the proper error and aborts the db transaction safely when a modifier callback fails. Fixes SF python patch/bug #1408584. - bsddb: multithreaded DB access using the simple bsddb module interface now works reliably. It has been updated to use automatic BerkeleyDB deadlock detection and the bsddb.dbutils.DeadlockWrap wrapper to retry database calls that would previously deadlock. [SF python bug #775414] - Patch #1446489: add support for the ZIP64 extensions to zipfile. - Patch #1506645: add Python wrappers for the curses functions is_term_resized, resize_term and resizeterm. Library ------- - Patch #815924: Restore ability to pass type= and icon= in tkMessageBox functions. - Patch #812986: Update turtle output even if not tracing. - Patch #1494750: Destroy master after deleting children in Tkinter.BaseWidget. - Patch #1096231: Add ``default`` argument to Tkinter.Wm.wm_iconbitmap. - Patch #763580: Add name and value arguments to Tkinter variable classes. - Bug #1117556: SimpleHTTPServer now tries to find and use the system's mime.types file for determining MIME types. - Bug #1339007: Shelf objects now don't raise an exception in their __del__ method when initialization failed. - Patch #1455898: The MBCS codec now supports the incremental mode for double-byte encodings. - ``difflib``'s ``SequenceMatcher.get_matching_blocks()`` was changed to guarantee that adjacent triples in the return list always describe non-adjacent blocks. Previously, a pair of matching blocks could end up being described by multiple adjacent triples that formed a partition of the matching pair. - Bug #1498146: fix optparse to handle Unicode strings in option help, description, and epilog. - Bug #1366250: minor optparse documentation error. - Bug #1361643: fix textwrap.dedent() so it handles tabs appropriately; clarify docs. - The wsgiref package has been added to the standard library. - The functions update_wrapper() and wraps() have been added to the functools module. These make it easier to copy relevant metadata from the original function when writing wrapper functions. - The optional ``isprivate`` argument to ``doctest.testmod()``, and the ``doctest.is_private()`` function, both deprecated in 2.4, were removed. - Patch #1359618: Speed up charmap encoder by using a trie structure for lookup. - The functions in the ``pprint`` module now sort dictionaries by key before computing the display. Before 2.5, ``pprint`` sorted a dictionary if and only if its display required more than one line, although that wasn't documented. The new behavior increases predictability; e.g., using ``pprint.pprint(a_dict)`` in a doctest is now reliable. - Patch #1497027: try HTTP digest auth before basic auth in urllib2 (thanks for J. J. Lee). - Patch #1496206: improve urllib2 handling of passwords with respect to default HTTP and HTTPS ports. - Patch #1080727: add "encoding" parameter to doctest.DocFileSuite. - Patch #1281707: speed up gzip.readline. - Patch #1180296: Two new functions were added to the locale module: format_string() to get the effect of "format % items" but locale-aware, and currency() to format a monetary number with currency sign. - Patch #1486962: Several bugs in the turtle Tk demo module were fixed and several features added, such as speed and geometry control. - Patch #1488881: add support for external file objects in bz2 compressed tarfiles. - Patch #721464: pdb.Pdb instances can now be given explicit stdin and stdout arguments, making it possible to redirect input and output for remote debugging. - Patch #1484695: Update the tarfile module to version 0.8. This fixes a couple of issues, notably handling of long file names using the GNU LONGNAME extension. - Patch #1478292. ``doctest.register_optionflag(name)`` shouldn't create a new flag when ``name`` is already the name of an option flag. - Bug #1385040: don't allow "def foo(a=1, b): pass" in the compiler package. - Patch #1472854: make the rlcompleter.Completer class usable on non- UNIX platforms. - Patch #1470846: fix urllib2 ProxyBasicAuthHandler. - Bug #1472827: correctly escape newlines and tabs in attribute values in the saxutils.XMLGenerator class. Build ----- - Bug #1502728: Correctly link against librt library on HP-UX. - OpenBSD 3.9 is supported now. - Patch #1492356: Port to Windows CE. - Bug/Patch #1481770: Use .so extension for shared libraries on HP-UX for ia64. - Patch #1471883: Add --enable-universalsdk. C API ----- Tests ----- Tools ----- Documentation ------------- What's New in Python 2.5 alpha 2? ================================= *Release date: 27-APR-2006* Core and builtins ----------------- - Bug #1465834: 'bdist_wininst preinstall script support' was fixed by converting these apis from macros into exported functions again: PyParser_SimpleParseFile PyParser_SimpleParseString PyRun_AnyFile PyRun_AnyFileEx PyRun_AnyFileFlags PyRun_File PyRun_FileEx PyRun_FileFlags PyRun_InteractiveLoop PyRun_InteractiveOne PyRun_SimpleFile PyRun_SimpleFileEx PyRun_SimpleString PyRun_String Py_CompileString - Under COUNT_ALLOCS, types are not necessarily immortal anymore. - All uses of PyStructSequence_InitType have been changed to initialize the type objects only once, even if the interpreter is initialized multiple times. - Bug #1454485, array.array('u') could crash the interpreter. This was due to PyArgs_ParseTuple(args, 'u#', ...) trying to convert buffers (strings) to unicode when it didn't make sense. 'u#' now requires a unicode string. - Py_UNICODE is unsigned. It was always documented as unsigned, but due to a bug had a signed value in previous versions. - Patch #837242: ``id()`` of any Python object always gives a positive number now, which might be a long integer. ``PyLong_FromVoidPtr`` and ``PyLong_AsVoidPtr`` have been changed accordingly. Note that it has never been correct to implement a ``__hash()__`` method that returns the ``id()`` of an object: def __hash__(self): return id(self) # WRONG because a hash result must be a (short) Python int but it was always possible for ``id()`` to return a Python long. However, because ``id()`` could return negative values before, on a 32-bit box an ``id()`` result was always usable as a hash value before this patch. That's no longer necessarily so. - Python on OS X 10.3 and above now uses dlopen() (via dynload_shlib.c) to load extension modules and now provides the dl module. As a result, sys.setdlopenflags() now works correctly on these systems. (SF patch #1454844) - Patch #1463867: enhanced garbage collection to allow cleanup of cycles involving generators that have paused outside of any ``try`` or ``with`` blocks. (In 2.5a1, a paused generator that was part of a reference cycle could not be garbage collected, regardless of whether it was paused in a ``try`` or ``with`` block.) Extension Modules ----------------- - Patch #1191065: Fix preprocessor problems on systems where recvfrom is a macro. - Bug #1467952: os.listdir() now correctly raises an error if readdir() fails with an error condition. - Fixed bsddb.db.DBError derived exceptions so they can be unpickled. - Bug #1117761: bsddb.*open() no longer raises an exception when using the cachesize parameter. - Bug #1149413: bsddb.*open() no longer raises an exception when using a temporary db (file=None) with the 'n' flag to truncate on open. - Bug #1332852: bsddb module minimum BerkeleyDB version raised to 3.3 as older versions cause excessive test failures. - Patch #1062014: AF_UNIX sockets under Linux have a special abstract namespace that is now fully supported. Library ------- - Bug #1223937: subprocess.CalledProcessError reports the exit status of the process using the returncode attribute, instead of abusing errno. - Patch #1475231: ``doctest`` has a new ``SKIP`` option, which causes a doctest to be skipped (the code is not run, and the expected output or exception is ignored). - Fixed contextlib.nested to cope with exceptions being raised and caught inside exit handlers. - Updated optparse module to Optik 1.5.1 (allow numeric constants in hex, octal, or binary; add ``append_const`` action; keep going if gettext cannot be imported; added ``OptionParser.destroy()`` method; added ``epilog`` for better help generation). - Bug #1473760: ``tempfile.TemporaryFile()`` could hang on Windows, when called from a thread spawned as a side effect of importing a module. - The pydoc module now supports documenting packages contained in .zip or .egg files. - The pkgutil module now has several new utility functions, such as ``walk_packages()`` to support working with packages that are either in the filesystem or zip files. - The mailbox module can now modify and delete messages from mailboxes, in addition to simply reading them. Thanks to Gregory K. Johnson for writing the code, and to the 2005 Google Summer of Code for funding his work. - The ``__del__`` method of class ``local`` in module ``_threading_local`` returned before accomplishing any of its intended cleanup. - Patch #790710: Add breakpoint command lists in pdb. - Patch #1063914: Add Tkinter.Misc.clipboard_get(). - Patch #1191700: Adjust column alignment in bdb breakpoint lists. - SimpleXMLRPCServer relied on the fcntl module, which is unavailable on Windows. Bug #1469163. - The warnings, linecache, inspect, traceback, site, and doctest modules were updated to work correctly with modules imported from zipfiles or via other PEP 302 __loader__ objects. - Patch #1467770: Reduce usage of subprocess._active to processes which the application hasn't waited on. - Patch #1462222: Fix Tix.Grid. - Fix exception when doing glob.glob('anything*/') - The pstats.Stats class accepts an optional stream keyword argument to direct output to an alternate file-like object. Build ----- - The Makefile now has a reindent target, which runs reindent.py on the library. - Patch #1470875: Building Python with MS Free Compiler - Patch #1161914: Add a python-config script. - Patch #1324762:Remove ccpython.cc; replace --with-cxx with --with-cxx-main. Link with C++ compiler only if --with-cxx-main was specified. (Can be overridden by explicitly setting LINKCC.) Decouple CXX from --with-cxx-main, see description in README. - Patch #1429775: Link extension modules with the shared libpython. - Fixed a libffi build problem on MIPS systems. - ``PyString_FromFormat``, ``PyErr_Format``, and ``PyString_FromFormatV`` now accept formats "%u" for unsigned ints, "%lu" for unsigned longs, and "%zu" for unsigned integers of type ``size_t``. Tests ----- - test_contextlib now checks contextlib.nested can cope with exceptions being raised and caught inside exit handlers. - test_cmd_line now checks operation of the -m and -c command switches - The test_contextlib test in 2.5a1 wasn't actually run unless you ran it separately and by hand. It also wasn't cleaning up its changes to the current Decimal context. - regrtest.py now has a -M option to run tests that test the new limits of containers, on 64-bit architectures. Running these tests is only sensible on 64-bit machines with more than two gigabytes of memory. The argument passed is the maximum amount of memory for the tests to use. Tools ----- - Added the Python benchmark suite pybench to the Tools/ directory; contributed by Marc-Andre Lemburg. Documentation ------------- - Patch #1473132: Improve docs for ``tp_clear`` and ``tp_traverse``. - PEP 343: Added Context Types section to the library reference and attempted to bring other PEP 343 related documentation into line with the implementation and/or python-dev discussions. - Bug #1337990: clarified that ``doctest`` does not support examples requiring both expected output and an exception. What's New in Python 2.5 alpha 1? ================================= *Release date: 05-APR-2006* Core and builtins ----------------- - PEP 338: -m command line switch now delegates to runpy.run_module allowing it to support modules in packages and zipfiles - On Windows, .DLL is not an accepted file name extension for extension modules anymore; extensions are only found if they end in .PYD. - Bug #1421664: sys.stderr.encoding is now set to the same value as sys.stdout.encoding. - __import__ accepts keyword arguments. - Patch #1460496: round() now accepts keyword arguments. - Fixed bug #1459029 - unicode reprs were double-escaped. - Patch #1396919: The system scope threads are reenabled on FreeBSD 5.4 and later versions. - Bug #1115379: Compiling a Unicode string with an encoding declaration now gives a SyntaxError. - Previously, Python code had no easy way to access the contents of a cell object. Now, a ``cell_contents`` attribute has been added (closes patch #1170323). - Patch #1123430: Python's small-object allocator now returns an arena to the system ``free()`` when all memory within an arena becomes unused again. Prior to Python 2.5, arenas (256KB chunks of memory) were never freed. Some applications will see a drop in virtual memory size now, especially long-running applications that, from time to time, temporarily use a large number of small objects. Note that when Python returns an arena to the platform C's ``free()``, there's no guarantee that the platform C library will in turn return that memory to the operating system. The effect of the patch is to stop making that impossible, and in tests it appears to be effective at least on Microsoft C and gcc-based systems. Thanks to Evan Jones for hard work and patience. - Patch #1434038: property() now uses the getter's docstring if there is no "doc" argument given. This makes it possible to legitimately use property() as a decorator to produce a read-only property. - PEP 357, patch 1436368: add an __index__ method to int/long and a matching nb_index slot to the PyNumberMethods struct. The slot is consulted instead of requiring an int or long in slicing and a few other contexts, enabling other objects (e.g. Numeric Python's integers) to be used as slice indices. - Fixed various bugs reported by Coverity's Prevent tool. - PEP 352, patch #1104669: Make exceptions new-style objects. Introduced the new exception base class, BaseException, which has a new message attribute. KeyboardInterrupt and SystemExit to directly inherit from BaseException now. Raising a string exception now raises a DeprecationWarning. - Patch #1438387, PEP 328: relative and absolute imports. Imports can now be explicitly relative, using 'from .module import name' to mean 'from the same package as this module is in. Imports without dots still default to the old relative-then-absolute, unless 'from __future__ import absolute_import' is used. - Properly check if 'warnings' raises an exception (usually when a filter set to "error" is triggered) when raising a warning for raising string exceptions. - CO_GENERATOR_ALLOWED is no longer defined. This behavior is the default. The name was removed from Include/code.h. - PEP 308: conditional expressions were added: (x if cond else y). - Patch 1433928: - The copy module now "copies" function objects (as atomic objects). - dict.__getitem__ now looks for a __missing__ hook before raising KeyError. - PEP 343: with statement implemented. Needs ``from __future__ import with_statement``. Use of 'with' as a variable will generate a warning. Use of 'as' as a variable will also generate a warning (unless it's part of an import statement). The following objects have __context__ methods: - The built-in file type. - The thread.LockType type. - The following types defined by the threading module: Lock, RLock, Condition, Semaphore, BoundedSemaphore. - The decimal.Context class. - Fix the encodings package codec search function to only search inside its own package. Fixes problem reported in patch #1433198. Note: Codec packages should implement and register their own codec search function. PEP 100 has the details. - PEP 353: Using ``Py_ssize_t`` as the index type. - ``PYMALLOC_DEBUG`` builds now add ``4*sizeof(size_t)`` bytes of debugging info to each allocated block, since the ``Py_ssize_t`` changes (PEP 353) now allow Python to make use of memory blocks exceeding 2**32 bytes for some purposes on 64-bit boxes. A ``PYMALLOC_DEBUG`` build was limited to 4-byte allocations before. - Patch #1400181, fix unicode string formatting to not use the locale. This is how string objects work. u'%f' could use , instead of . for the decimal point. Now both strings and unicode always use periods. - Bug #1244610, #1392915, fix build problem on OpenBSD 3.7 and 3.8. configure would break checking curses.h. - Bug #959576: The pwd module is now builtin. This allows Python to be built on UNIX platforms without $HOME set. - Bug #1072182, fix some potential problems if characters are signed. - Bug #889500, fix line number on SyntaxWarning for global declarations. - Bug #1378022, UTF-8 files with a leading BOM crashed the interpreter. - Support for converting hex strings to floats no longer works. This was not portable. float('0x3') now raises a ValueError. - Patch #1382163: Expose Subversion revision number to Python. New C API function Py_GetBuildNumber(). New attribute sys.subversion. Build number is now displayed in interactive prompt banner. - Implementation of PEP 341 - Unification of try/except and try/finally. "except" clauses can now be written together with a "finally" clause in one try statement instead of two nested ones. Patch #1355913. - Bug #1379994: Builtin unicode_escape and raw_unicode_escape codec now encodes backslash correctly. - Patch #1350409: Work around signal handling bug in Visual Studio 2005. - Bug #1281408: Py_BuildValue now works correctly even with unsigned longs and long longs. - SF Bug #1350188, "setdlopenflags" leads to crash upon "import" It was possible for dlerror() to return a NULL pointer, so it will now use a default error message in this case. - Replaced most Unicode charmap codecs with new ones using the new Unicode translate string feature in the builtin charmap codec; the codecs were created from the mapping tables available at ftp.unicode.org and contain a few updates (e.g. the Mac OS encodings now include a mapping for the Apple logo) - Added a few more codecs for Mac OS encodings - Sped up some Unicode operations. - A new AST parser implementation was completed. The abstract syntax tree is available for read-only (non-compile) access to Python code; an _ast module was added. - SF bug #1167751: fix incorrect code being produced for generator expressions. The following code now raises a SyntaxError: foo(a = i for i in range(10)) - SF Bug #976608: fix SystemError when mtime of an imported file is -1. - SF Bug #887946: fix segfault when redirecting stdin from a directory. Provide a warning when a directory is passed on the command line. - Fix segfault with invalid coding. - SF bug #772896: unknown encoding results in MemoryError. - All iterators now have a Boolean value of True. Formerly, some iterators supported a __len__() method which evaluated to False when the iterator was empty. - On 64-bit platforms, when __len__() returns a value that cannot be represented as a C int, raise OverflowError. - test__locale is skipped on OS X < 10.4 (only partial locale support is present). - SF bug #893549: parsing keyword arguments was broken with a few format codes. - Changes donated by Elemental Security to make it work on AIX 5.3 with IBM's 64-bit compiler (SF patch #1284289). This also closes SF bug #105470: test_pwd fails on 64bit system (Opteron). - Changes donated by Elemental Security to make it work on HP-UX 11 on Itanium2 with HP's 64-bit compiler (SF patch #1225212). - Disallow keyword arguments for type constructors that don't use them (fixes bug #1119418). - Forward UnicodeDecodeError into SyntaxError for source encoding errors. - SF bug #900092: When tracing (e.g. for hotshot), restore 'return' events for exceptions that cause a function to exit. - The implementation of set() and frozenset() was revised to use its own internal data structure. Memory consumption is reduced by 1/3 and there are modest speed-ups as well. The API is unchanged. - SF bug #1238681: freed pointer is used in longobject.c:long_pow(). - SF bug #1229429: PyObject_CallMethod failed to decrement some reference counts in some error exit cases. - SF bug #1185883: Python's small-object memory allocator took over a block managed by the platform C library whenever a realloc specified a small new size. However, there's no portable way to know then how much of the address space following the pointer is valid, so there's no portable way to copy data from the C-managed block into Python's small-object space without risking a memory fault. Python's small-object realloc now leaves such blocks under the control of the platform C realloc. - SF bug #1232517: An overflow error was not detected properly when attempting to convert a large float to an int in os.utime(). - SF bug #1224347: hex longs now print with lowercase letters just like their int counterparts. - SF bug #1163563: the original fix for bug #1010677 ("thread Module Breaks PyGILState_Ensure()") broke badly in the case of multiple interpreter states; back out that fix and do a better job (see http://mail.python.org/pipermail/python-dev/2005-June/054258.html for a longer write-up of the problem). - SF patch #1180995: marshal now uses a binary format by default when serializing floats. - SF patch #1181301: on platforms that appear to use IEEE 754 floats, the routines that promise to produce IEEE 754 binary representations of floats now simply copy bytes around. - bug #967182: disallow opening files with 'wU' or 'aU' as specified by PEP 278. - patch #1109424: int, long, float, complex, and unicode now check for the proper magic slot for type conversions when subclassed. Previously the magic slot was ignored during conversion. Semantics now match the way subclasses of str always behaved. int/long/float, conversion of an instance to the base class has been moved to the proper nb_* magic slot and out of PyNumber_*(). Thanks Walter D�rwald. - Descriptors defined in C with a PyGetSetDef structure, where the setter is NULL, now raise an AttributeError when attempting to set or delete the attribute. Previously a TypeError was raised, but this was inconsistent with the equivalent pure-Python implementation. - It is now safe to call PyGILState_Release() before PyEval_InitThreads() (note that if there is reason to believe there are multiple threads around you still must call PyEval_InitThreads() before using the Python API; this fix is for extension modules that have no way of knowing if Python is multi-threaded yet). - Typing Ctrl-C whilst raw_input() was waiting in a build with threads disabled caused a crash. - Bug #1165306: instancemethod_new allowed the creation of a method with im_class == im_self == NULL, which caused a crash when called. - Move exception finalisation later in the shutdown process - this fixes the crash seen in bug #1165761 - Added two new builtins, any() and all(). - Defining a class with empty parentheses is now allowed (e.g., ``class C(): pass`` is no longer a syntax error). Patch #1176012 added support to the 'parser' module and 'compiler' package (thanks to logistix for that added support). - Patch #1115086: Support PY_LONGLONG in structmember. - Bug #1155938: new style classes did not check that __init__() was returning None. - Patch #802188: Report characters after line continuation character ('\') with a specific error message. - Bug #723201: Raise a TypeError for passing bad objects to 'L' format. - Bug #1124295: the __name__ attribute of file objects was inadvertently made inaccessible in restricted mode. - Bug #1074011: closing sys.std{out,err} now causes a flush() and an ferror() call. - min() and max() now support key= arguments with the same meaning as in list.sort(). - The peephole optimizer now performs simple constant folding in expressions: (2+3) --> (5). - set and frozenset objects can now be marshalled. SF #1098985. - Bug #1077106: Poor argument checking could cause memory corruption in calls to os.read(). - The parser did not complain about future statements in illegal positions. It once again reports a syntax error if a future statement occurs after anything other than a doc string. - Change the %s format specifier for str objects so that it returns a unicode instance if the argument is not an instance of basestring and calling __str__ on the argument returns a unicode instance. - Patch #1413181: changed ``PyThreadState_Delete()`` to forget about the current thread state when the auto-GIL-state machinery knows about it (since the thread state is being deleted, continuing to remember it can't help, but can hurt if another thread happens to get created with the same thread id). Extension Modules ----------------- - Patch #1380952: fix SSL objects timing out on consecutive read()s - Patch #1309579: wait3 and wait4 were added to the posix module. - Patch #1231053: The audioop module now supports encoding/decoding of alaw. In addition, the existing ulaw code was updated. - RFE #567972: Socket objects' family, type and proto properties are now exposed via new attributes. - Everything under lib-old was removed. This includes the following modules: Para, addpack, cmp, cmpcache, codehack, dircmp, dump, find, fmt, grep, lockfile, newdir, ni, packmail, poly, rand, statcache, tb, tzparse, util, whatsound, whrandom, zmod - The following modules were removed: regsub, reconvert, regex, regex_syntax. - re and sre were swapped, so help(re) provides full help. importing sre is deprecated. The undocumented re.engine variable no longer exists. - Bug #1448490: Fixed a bug that ISO-2022 codecs could not handle SS2 (single-shift 2) escape sequences correctly. - The unicodedata module was updated to the 4.1 version of the Unicode database. The 3.2 version is still available as unicodedata.db_3_2_0 for applications that require this specific version (such as IDNA). - The timing module is no longer built by default. It was deprecated in PEP 4 in Python 2.0 or earlier. - Patch 1433928: Added a new type, defaultdict, to the collections module. This uses the new __missing__ hook behavior added to dict (see above). - Bug #854823: socketmodule now builds on Sun platforms even when INET_ADDRSTRLEN is not defined. - Patch #1393157: os.startfile() now has an optional argument to specify a "command verb" to invoke on the file. - Bug #876637, prevent stack corruption when socket descriptor is larger than FD_SETSIZE. - Patch #1407135, bug #1424041: harmonize mmap behavior of anonymous memory. mmap.mmap(-1, size) now returns anonymous memory in both Unix and Windows. mmap.mmap(0, size) should not be used on Windows for anonymous memory. - Patch #1422385: The nis module now supports access to domains other than the system default domain. - Use Win32 API to implement os.stat/fstat. As a result, subsecond timestamps are reported, the limit on path name lengths is removed, and stat reports WindowsError now (instead of OSError). - Add bsddb.db.DBEnv.set_tx_timestamp allowing time based database recovery. - Bug #1413192, fix seg fault in bsddb if a transaction was deleted before the env. - Patch #1103116: Basic AF_NETLINK support. - Bug #1402308, (possible) segfault when using mmap.mmap(-1, ...) - Bug #1400822, _curses over{lay,write} doesn't work when passing 6 ints. Also fix ungetmouse() which did not accept arguments properly. The code now conforms to the documented signature. - Bug #1400115, Fix segfault when calling curses.panel.userptr() without prior setting of the userptr. - Fix 64-bit problems in bsddb. - Patch #1365916: fix some unsafe 64-bit mmap methods. - Bug #1290333: Added a workaround for cjkcodecs' _codecs_cn build problem on AIX. - Bug #869197: os.setgroups rejects long integer arguments - Bug #1346533, select.poll() doesn't raise an error if timeout > sys.maxint - Bug #1344508, Fix UNIX mmap leaking file descriptors - Patch #1338314, Bug #1336623: fix tarfile so it can extract REGTYPE directories from tarfiles written by old programs. - Patch #1407992, fixes broken bsddb module db associate when using BerkeleyDB 3.3, 4.0 or 4.1. - Get bsddb module to build with BerkeleyDB version 4.4 - Get bsddb module to build with BerkeleyDB version 3.2 - Patch #1309009, Fix segfault in pyexpat when the XML document is in latin_1, but Python incorrectly assumes it is in UTF-8 format - Fix parse errors in the readline module when compiling without threads. - Patch #1288833: Removed thread lock from socket.getaddrinfo on FreeBSD 5.3 and later versions which got thread-safe getaddrinfo(3). - Patches #1298449 and #1298499: Add some missing checks for error returns in cStringIO.c. - Patch #1297028: fix segfault if call type on MultibyteCodec, MultibyteStreamReader, or MultibyteStreamWriter - Fix memory leak in posix.access(). - Patch #1213831: Fix typo in unicodedata._getcode. - Bug #1007046: os.startfile() did not accept unicode strings encoded in the file system encoding. - Patch #756021: Special-case socket.inet_aton('255.255.255.255') for platforms that don't have inet_aton(). - Bug #1215928: Fix bz2.BZ2File.seek() for 64-bit file offsets. - Bug #1191043: Fix bz2.BZ2File.(x)readlines for files containing one line without newlines. - Bug #728515: mmap.resize() now resizes the file on Unix as it did on Windows. - Patch #1180695: Add nanosecond stat resolution, and st_gen, st_birthtime for FreeBSD. - Patch #1231069: The fcntl.ioctl function now uses the 'I' code for the request code argument, which results in more C-like behaviour for large or negative values. - Bug #1234979: For the argument of thread.Lock.acquire, the Windows implementation treated all integer values except 1 as false. - Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. - Patch #1212117: os.stat().st_flags is now accessible as a attribute if available on the platform. - Patch #1103951: Expose O_SHLOCK and O_EXLOCK in the posix module if available on the platform. - Bug #1166660: The readline module could segfault if hook functions were set in a different thread than that which called readline. - collections.deque objects now support a remove() method. - operator.itemgetter() and operator.attrgetter() now support retrieving multiple fields. This provides direct support for sorting on multiple keys (primary, secondary, etc). - os.access now supports Unicode path names on non-Win32 systems. - Patches #925152, #1118602: Avoid reading after the end of the buffer in pyexpat.GetInputContext. - Patches #749830, #1144555: allow UNIX mmap size to default to current file size. - Added functional.partial(). See PEP309. - Patch #1093585: raise a ValueError for negative history items in readline. {remove_history,replace_history} - The spwd module has been added, allowing access to the shadow password database. - stat_float_times is now True. - array.array objects are now picklable. - the cPickle module no longer accepts the deprecated None option in the args tuple returned by __reduce__(). - itertools.islice() now accepts None for the start and step arguments. This allows islice() to work more readily with slices: islice(s.start, s.stop, s.step) - datetime.datetime() now has a strptime class method which can be used to create datetime object using a string and format. - Patch #1117961: Replace the MD5 implementation from RSA Data Security Inc with the implementation from http://sourceforge.net/projects/libmd5-rfc/. Library ------- - Patch #1388073: Numerous __-prefixed attributes of unittest.TestCase have been renamed to have only a single underscore prefix. This was done to make subclassing easier. - PEP 338: new module runpy defines a run_module function to support executing modules which provide access to source code or a code object via the PEP 302 import mechanisms. - The email module's parsedate_tz function now sets the daylight savings flag to -1 (unknown) since it can't tell from the date whether it should be set. - Patch #624325: urlparse.urlparse() and urlparse.urlsplit() results now sport attributes that provide access to the parts of the result. - Patch #1462498: sgmllib now handles entity and character references in attribute values. - Added the sqlite3 package. This is based on pysqlite2.1.3, and provides a DB-API interface in the standard library. You'll need sqlite 3.0.8 or later to build this - if you have an earlier version, the C extension module will not be built. - Bug #1460340: ``random.sample(dict)`` failed in various ways. Dicts aren't officially supported here, and trying to use them will probably raise an exception some day. But dicts have been allowed, and "mostly worked", so support for them won't go away without warning. - Bug #1445068: getpass.getpass() can now be given an explicit stream argument to specify where to write the prompt. - Patch #1462313, bug #1443328: the pickle modules now can handle classes that have __private names in their __slots__. - Bug #1250170: mimetools now handles socket.gethostname() failures gracefully. - patch #1457316: "setup.py upload" now supports --identity to select the key to be used for signing the uploaded code. - Queue.Queue objects now support .task_done() and .join() methods to make it easier to monitor when daemon threads have completed processing all enqueued tasks. Patch #1455676. - popen2.Popen objects now preserve the command in a .cmd attribute. - Added the ctypes ffi package. - email 4.0 package now integrated. This is largely the same as the email 3.0 package that was included in Python 2.3, except that PEP 8 module names are now used (e.g. mail.message instead of email.Message). The MIME classes have been moved to a subpackage (e.g. email.mime.text instead of email.MIMEText). The old names are still supported for now. Several deprecated Message methods have been removed and lots of bugs have been fixed. More details can be found in the email package documentation. - Patches #1436130/#1443155: codecs.lookup() now returns a CodecInfo object (a subclass of tuple) that provides incremental decoders and encoders (a way to use stateful codecs without the stream API). Python functions codecs.getincrementaldecoder() and codecs.getincrementalencoder() as well as C functions PyCodec_IncrementalEncoder() and PyCodec_IncrementalDecoder() have been added. - Patch #1359365: Calling next() on a closed StringIO.String object raises a ValueError instead of a StopIteration now (like file and cString.String do). cStringIO.StringIO.isatty() will raise a ValueError now if close() has been called before (like file and StringIO.StringIO do). - A regrtest option -w was added to re-run failed tests in verbose mode. - Patch #1446372: quit and exit can now be called from the interactive interpreter to exit. - The function get_count() has been added to the gc module, and gc.collect() grew an optional 'generation' argument. - A library msilib to generate Windows Installer files, and a distutils command bdist_msi have been added. - PEP 343: new module contextlib.py defines decorator @contextmanager and helpful context managers nested() and closing(). - The compiler package now supports future imports after the module docstring. - Bug #1413790: zipfile now sanitizes absolute archive names that are not allowed by the specs. - Patch #1215184: FileInput now can be given an opening hook which can be used to control how files are opened. - Patch #1212287: fileinput.input() now has a mode parameter for specifying the file mode input files should be opened with. - Patch #1215184: fileinput now has a fileno() function for getting the current file number. - Patch #1349274: gettext.install() now optionally installs additional translation functions other than _() in the builtin namespace. - Patch #1337756: fileinput now accepts Unicode filenames. - Patch #1373643: The chunk module can now read chunks larger than two gigabytes. - Patch #1417555: SimpleHTTPServer now returns Last-Modified headers. - Bug #1430298: It is now possible to send a mail with an empty return address using smtplib. - Bug #1432260: The names of lambda functions are now properly displayed in pydoc. - Patch #1412872: zipfile now sets the creator system to 3 (Unix) unless the system is Win32. - Patch #1349118: urllib now supports user:pass@ style proxy specifications, raises IOErrors when proxies for unsupported protocols are defined, and uses the https proxy on https redirections. - Bug #902075: urllib2 now supports 'host:port' style proxy specifications. - Bug #1407902: Add support for sftp:// URIs to urlparse. - Bug #1371247: Update Windows locale identifiers in locale.py. - Bug #1394565: SimpleHTTPServer now doesn't choke on query parameters any more. - Bug #1403410: The warnings module now doesn't get confused when it can't find out the module name it generates a warning for. - Patch #1177307: Added a new codec utf_8_sig for UTF-8 with a BOM signature. - Patch #1157027: cookielib mishandles RFC 2109 cookies in Netscape mode - Patch #1117398: cookielib.LWPCookieJar and .MozillaCookieJar now raise LoadError as documented, instead of IOError. For compatibility, LoadError subclasses IOError. - Added the hashlib module. It provides secure hash functions for MD5 and SHA1, 224, 256, 384, and 512. Note that recent developments make the historic MD5 and SHA1 unsuitable for cryptographic-strength applications. In Ronald L. Rivest offered this advice for Python: "The consensus of researchers in this area (at least as expressed at the NIST Hash Function Workshop 10/31/05), is that SHA-256 is a good choice for the time being, but that research should continue, and other alternatives may arise from this research. The larger SHA's also seem OK." - Added a subset of Fredrik Lundh's ElementTree package. Available modules are xml.etree.ElementTree, xml.etree.ElementPath, and xml.etree.ElementInclude, from ElementTree 1.2.6. - Patch #1162825: Support non-ASCII characters in IDLE window titles. - Bug #1365984: urllib now opens "data:" URLs again. - Patch #1314396: prevent deadlock for threading.Thread.join() when an exception is raised within the method itself on a previous call (e.g., passing in an illegal argument) - Bug #1340337: change time.strptime() to always return ValueError when there is an error in the format string. - Patch #754022: Greatly enhanced webbrowser.py (by Oleg Broytmann). - Bug #729103: pydoc.py: Fix docother() method to accept additional "parent" argument. - Patch #1300515: xdrlib.py: Fix pack_fstring() to really use null bytes for padding. - Bug #1296004: httplib.py: Limit maximal amount of data read from the socket to avoid a MemoryError on Windows. - Patch #1166948: locale.py: Prefer LC_ALL, LC_CTYPE and LANG over LANGUAGE to get the correct encoding. - Patch #1166938: locale.py: Parse LANGUAGE as a colon separated list of languages. - Patch #1268314: Cache lines in StreamReader.readlines for performance. - Bug #1290505: Fix clearing the regex cache for time.strptime(). - Bug #1167128: Fix size of a symlink in a tarfile to be 0. - Patch #810023: Fix off-by-one bug in urllib.urlretrieve reporthook functionality. - Bug #1163178: Make IDNA return an empty string when the input is empty. - Patch #848017: Make Cookie more RFC-compliant. Use CRLF as default output separator and do not output trailing semicolon. - Patch #1062060: urllib.urlretrieve() now raises a new exception, named ContentTooShortException, when the actually downloaded size does not match the Content-Length header. - Bug #1121494: distutils.dir_utils.mkpath now accepts Unicode strings. - Bug #1178484: Return complete lines from codec stream readers even if there is an exception in later lines, resulting in correct line numbers for decoding errors in source code. - Bug #1192315: Disallow negative arguments to clear() in pdb. - Patch #827386: Support absolute source paths in msvccompiler.py. - Patch #1105730: Apply the new implementation of commonprefix in posixpath to ntpath, macpath, os2emxpath and riscospath. - Fix a problem in Tkinter introduced by SF patch #869468: delete bogus __hasattr__ and __delattr__ methods on class Tk that were breaking Tkdnd. - Bug #1015140: disambiguated the term "article id" in nntplib docs and docstrings to either "article number" or "message id". - Bug #1238170: threading.Thread.__init__ no longer has "kwargs={}" as a parameter, but uses the usual "kwargs=None". - textwrap now processes text chunks at O(n) speed instead of O(n**2). Patch #1209527 (Contributed by Connelly). - urllib2 has now an attribute 'httpresponses' mapping from HTTP status code to W3C name (404 -> 'Not Found'). RFE #1216944. - Bug #1177468: Don't cache the /dev/urandom file descriptor for os.urandom, as this can cause problems with apps closing all file descriptors. - Bug #839151: Fix an attempt to access sys.argv in the warnings module; it can be missing in embedded interpreters - Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via error messages. - Bug #1108948: Cookie.py produced invalid JavaScript code. - The tokenize module now detects and reports indentation errors. Bug #1224621. - The tokenize module has a new untokenize() function to support a full roundtrip from lexed tokens back to Python source code. In addition, the generate_tokens() function now accepts a callable argument that terminates by raising StopIteration. - Bug #1196315: fix weakref.WeakValueDictionary constructor. - Bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. - Patch #1120353: The xmlrpclib module provides better, more transparent, support for datetime.{datetime,date,time} objects. With use_datetime set to True, applications shouldn't have to fiddle with the DateTime wrapper class at all. - distutils.commands.upload was added to support uploading distribution files to PyPI. - distutils.commands.register now encodes the data as UTF-8 before posting them to PyPI. - decimal operator and comparison methods now return NotImplemented instead of raising a TypeError when interacting with other types. This allows other classes to implement __radd__ style methods and have them work as expected. - Bug #1163325: Decimal infinities failed to hash. Attempting to hash a NaN raised an InvalidOperation instead of a TypeError. - Patch #918101: Add tarfile open mode r|* for auto-detection of the stream compression; add, for symmetry reasons, r:* as a synonym of r. - Patch #1043890: Add extractall method to tarfile. - Patch #1075887: Don't require MSVC in distutils if there is nothing to build. - Patch #1103407: Properly deal with tarfile iterators when untarring symbolic links on Windows. - Patch #645894: Use getrusage for computing the time consumption in profile.py if available. - Patch #1046831: Use get_python_version where appropriate in sysconfig.py. - Patch #1117454: Remove code to special-case cookies without values in LWPCookieJar. - Patch #1117339: Add cookielib special name tests. - Patch #1112812: Make bsddb/__init__.py more friendly for modulefinder. - Patch #1110248: SYNC_FLUSH the zlib buffer for GZipFile.flush. - Patch #1107973: Allow to iterate over the lines of a tarfile.ExFileObject. - Patch #1104111: Alter setup.py --help and --help-commands. - Patch #1121234: Properly cleanup _exit and tkerror commands. - Patch #1049151: xdrlib now unpacks booleans as True or False. - Fixed bug in a NameError bug in cookielib. Patch #1116583. - Applied a security fix to SimpleXMLRPCserver (PSF-2005-001). This disables recursive traversal through instance attributes, which can be exploited in various ways. - Bug #1222790: in SimpleXMLRPCServer, set the reuse-address and close-on-exec flags on the HTTP listening socket. - Bug #792570: SimpleXMLRPCServer had problems if the request grew too large. Fixed by reading the HTTP body in chunks instead of one big socket.read(). - Patches #893642, #1039083: add allow_none, encoding arguments to constructors of SimpleXMLRPCServer and CGIXMLRPCRequestHandler. - Bug #1110478: Revert os.environ.update to do putenv again. - Bug #1103844: fix distutils.install.dump_dirs() with negated options. - os.{SEEK_SET, SEEK_CUR, SEEK_END} have been added for convenience. - Enhancements to the csv module: + Dialects are now validated by the underlying C code, better reflecting its capabilities, and improving its compliance with PEP 305. + Dialect parameter parsing has been re-implemented to improve error reporting. + quotechar=None and quoting=QUOTE_NONE now work the way PEP 305 dictates. + the parser now removes the escapechar prefix from escaped characters. + when quoting=QUOTE_NONNUMERIC, the writer now tests for numeric types, rather than any object that can be represented as a numeric. + when quoting=QUOTE_NONNUMERIC, the reader now casts unquoted fields to floats. + reader now allows \r characters to be quoted (previously it only allowed \n to be quoted). + writer doublequote handling improved. + Dialect classes passed to the module are no longer instantiated by the module before being parsed (the former validation scheme required this, but the mechanism was unreliable). + The dialect registry now contains instances of the internal C-coded dialect type, rather than references to python objects. + the internal c-coded dialect type is now immutable. + register_dialect now accepts the same keyword dialect specifications as the reader and writer, allowing the user to register dialects without first creating a dialect class. + a configurable limit to the size of parsed fields has been added - previously, an unmatched quote character could result in the entire file being read into the field buffer before an error was reported. + A new module method csv.field_size_limit() has been added that sets the parser field size limit (returning the former limit). The initial limit is 128kB. + A line_num attribute has been added to the reader object, which tracks the number of lines read from the source iterator. This is not the same as the number of records returned, as records can span multiple lines. + reader and writer objects were not being registered with the cyclic-GC. This has been fixed. - _DummyThread objects in the threading module now delete self.__block that is inherited from _Thread since it uses up a lock allocated by 'thread'. The lock primitives tend to be limited in number and thus should not be wasted on a _DummyThread object. Fixes bug #1089632. - The imghdr module now detects Exif files. - StringIO.truncate() now correctly adjusts the size attribute. (Bug #951915). - locale.py now uses an updated locale alias table (built using Tools/i18n/makelocalealias.py, a tool to parse the X11 locale alias file); the encoding lookup was enhanced to use Python's encoding alias table. - moved deprecated modules to Lib/lib-old: whrandom, tzparse, statcache. - the pickle module no longer accepts the deprecated None option in the args tuple returned by __reduce__(). - optparse now optionally imports gettext. This allows its use in setup.py. - the pickle module no longer uses the deprecated bin parameter. - the shelve module no longer uses the deprecated binary parameter. - the pstats module no longer uses the deprecated ignore() method. - the filecmp module no longer uses the deprecated use_statcache argument. - unittest.TestCase.run() and unittest.TestSuite.run() can now be successfully extended or overridden by subclasses. Formerly, the subclassed method would be ignored by the rest of the module. (Bug #1078905). - heapq.nsmallest() and heapq.nlargest() now support key= arguments with the same meaning as in list.sort(). - Bug #1076985: ``codecs.StreamReader.readline()`` now calls ``read()`` only once when a size argument is given. This prevents a buffer overflow in the tokenizer with very long source lines. - Bug #1083110: ``zlib.decompress.flush()`` would segfault if called immediately after creating the object, without any intervening ``.decompress()`` calls. - The reconvert.quote function can now emit triple-quoted strings. The reconvert module now has some simple documentation. - ``UserString.MutableString`` now supports negative indices in ``__setitem__`` and ``__delitem__`` - Bug #1149508: ``textwrap`` now handles hyphenated numbers (eg. "2004-03-05") correctly. - Partial fixes for SF bugs #1163244 and #1175396: If a chunk read by ``codecs.StreamReader.readline()`` has a trailing "\r", read one more character even if the user has passed a size parameter to get a proper line ending. Remove the special handling of a "\r\n" that has been split between two lines. - Bug #1251300: On UCS-4 builds the "unicode-internal" codec will now complain about illegal code points. The codec now supports PEP 293 style error handlers. - Bug #1235646: ``codecs.StreamRecoder.next()`` now reencodes the data it reads from the input stream, so that the output is a byte string in the correct encoding instead of a unicode string. - Bug #1202493: Fixing SRE parser to handle '{}' as perl does, rather than considering it exactly like a '*'. - Bug #1245379: Add "unicode-1-1-utf-7" as an alias for "utf-7" to ``encodings.aliases``. - ` uu.encode()`` and ``uu.decode()`` now support unicode filenames. - Patch #1413711: Certain patterns of differences were making difflib touch the recursion limit. - Bug #947906: An object oriented interface has been added to the calendar module. It's possible to generate HTML calendar now and the module can be called as a script (e.g. via ``python -mcalendar``). Localized month and weekday names can be ouput (even if an exotic encoding is used) using special classes that use unicode. Build ----- - Fix test_float, test_long, and test_struct failures on Tru64 with gcc by using -mieee gcc option. - Patch #1432345: Make python compile on DragonFly. - Build support for Win64-AMD64 was added. - Patch #1428494: Prefer linking against ncursesw over ncurses library. - Patch #881820: look for openpty and forkpty also in libbsd. - The sources of zlib are now part of the Python distribution (zlib 1.2.3). The zlib module is now builtin on Windows. - Use -xcode=pic32 for CCSHARED on Solaris with SunPro. - Bug #1189330: configure did not correctly determine the necessary value of LINKCC if python was built with GCC 4.0. - Upgrade Windows build to zlib 1.2.3 which eliminates a potential security vulnerability in zlib 1.2.1 and 1.2.2. - EXTRA_CFLAGS has been introduced as an environment variable to hold compiler flags that change binary compatibility. Changes were also made to distutils.sysconfig to also use the environment variable when used during compilation of the interpreter and of C extensions through distutils. - SF patch 1171735: Darwin 8's headers are anal about POSIX compliance, and linking has changed (prebinding is now deprecated, and libcc_dynamic no longer exists). This configure patch makes things right. - Bug #1158607: Build with --disable-unicode again. - spwdmodule.c is built only if either HAVE_GETSPNAM or HAVE_HAVE_GETSPENT is defined. Discovered as a result of not being able to build on OS X. - setup.py now uses the directories specified in LDFLAGS using the -L option and in CPPFLAGS using the -I option for adding library and include directories, respectively, for compiling extension modules against. This has led to the core being compiled using the values in CPPFLAGS. It also removes the need for the special-casing of both DarwinPorts and Fink for darwin since the proper directories can be specified in LDFLAGS (``-L/sw/lib`` for Fink, ``-L/opt/local/lib`` for DarwinPorts) and CPPFLAGS (``-I/sw/include`` for Fink, ``-I/opt/local/include`` for DarwinPorts). - Test in configure.in that checks for tzset no longer dependent on tm->tm_zone to exist in the struct (not required by either ISO C nor the UNIX 2 spec). Tests for sanity in tzname when HAVE_TZNAME defined were also defined. Closes bug #1096244. Thanks Gregory Bond. C API ----- - ``PyMem_{Del, DEL}`` and ``PyMem_{Free, FREE}`` no longer map to ``PyObject_{Free, FREE}``. They map to the system ``free()`` now. If memory is obtained via the ``PyObject_`` family, it must be released via the ``PyObject_`` family, and likewise for the ``PyMem_`` family. This has always been officially true, but when Python's small-object allocator was introduced, an attempt was made to cater to a few extension modules discovered at the time that obtained memory via ``PyObject_New`` but released it via ``PyMem_DEL``. It's years later, and if such code still exists it will fail now (probably with segfaults, but calling wrong low-level memory management functions can yield many symptoms). - Added a C API for set and frozenset objects. - Removed PyRange_New(). - Patch #1313939: PyUnicode_DecodeCharmap() accepts a unicode string as the mapping argument now. This string is used as a mapping table. Byte values greater than the length of the string and 0xFFFE are treated as undefined mappings. Tests ----- - In test_os, st_?time is now truncated before comparing it with ST_?TIME. - Patch #1276356: New resource "urlfetch" is implemented. This enables even impatient people to run tests that require remote files. Documentation ------------- - Bug #1402224: Add warning to dl docs about crashes. - Bug #1396471: Document that Windows' ftell() can return invalid values for text files with UNIX-style line endings. - Bug #1274828: Document os.path.splitunc(). - Bug #1190204: Clarify which directories are searched by site.py. - Bug #1193849: Clarify os.path.expanduser() documentation. - Bug #1243192: re.UNICODE and re.LOCALE affect \d, \D, \s and \S. - Bug #755617: Document the effects of os.chown() on Windows. - Patch #1180012: The documentation for modulefinder is now in the library reference. - Patch #1213031: Document that os.chown() accepts argument values of -1. - Bug #1190563: Document os.waitpid() return value with WNOHANG flag. - Bug #1175022: Correct the example code for property(). - Document the IterableUserDict class in the UserDict module. Closes bug #1166582. - Remove all latent references for "Macintosh" that referred to semantics for Mac OS 9 and change to reflect the state for OS X. Closes patch #1095802. Thanks Jack Jansen. Mac --- New platforms ------------- - FreeBSD 7 support is added. Tools/Demos ----------- - Created Misc/Vim/vim_syntax.py to auto-generate a python.vim file in that directory for syntax highlighting in Vim. Vim directory was added and placed vimrc to it (was previous up a level). - Added two new files to Tools/scripts: pysource.py, which recursively finds Python source files, and findnocoding.py, which finds Python source files that need an encoding declaration. Patch #784089, credits to Oleg Broytmann. - Bug #1072853: pindent.py used an uninitialized variable. - Patch #1177597: Correct Complex.__init__. - Fixed a display glitch in Pynche, which could cause the right arrow to wiggle over by a pixel. What's New in Python 2.4 final? =============================== *Release date: 30-NOV-2004* Core and builtins ----------------- - Bug 875692: Improve signal handling, especially when using threads, by forcing an early re-execution of PyEval_EvalFrame() "periodic" code when things_to_do is not cleared by Py_MakePendingCalls(). What's New in Python 2.4 (release candidate 1) ============================================== *Release date: 18-NOV-2004* Core and builtins ----------------- - Bug 1061968: Fixes in 2.4a3 to address thread bug 1010677 reintroduced the years-old thread shutdown race bug 225673. Numeric history lesson aside, all bugs in all three reports are fixed now. Library ------- - Bug 1052242: If exceptions are raised by an atexit handler function an attempt is made to execute the remaining handlers. The last exception raised is re-raised. - ``doctest``'s new support for adding ``pdb.set_trace()`` calls to doctests was broken in a dramatic but shallow way. Fixed. - Bug 1065388: ``calendar``'s ``day_name``, ``day_abbr``, ``month_name``, and ``month_abbr`` attributes emulate sequences of locale-correct spellings of month and day names. Because the locale can change at any time, the correct spelling is recomputed whenever one of these is indexed. In the worst case, the index may be a slice object, so these recomputed every day or month name each time they were indexed. This is much slower than necessary in the usual case, when the index is just an integer. In that case, only the single spelling needed is recomputed now; and, when the index is a slice object, only the spellings needed by the slice are recomputed now. - Patch 1061679: Added ``__all__`` to pickletools.py. Build ----- - Bug 1034277 / Patch 1035255: Remove compilation of core against CoreServices and CoreFoundation on OS X. Involved removing PyMac_GetAppletScriptFile() which has no known users. Thanks Bob Ippolito. C API ----- - The PyRange_New() function is deprecated. What's New in Python 2.4 beta 2? ================================ *Release date: 03-NOV-2004* License ------- The Python Software Foundation changed the license under which Python is released, to remove Python version numbers. There were no other changes to the license. So, for example, wherever the license for Python 2.3 said "Python 2.3", the new license says "Python". The intent is to make it possible to refer to the PSF license in a more durable way. For example, some people say they're confused by that the Open Source Initiative's entry for the Python Software Foundation License:: http://www.opensource.org/licenses/PythonSoftFoundation.php says "Python 2.1.1" all over it, wondering whether it applies only to Python 2.1.1. The official name of the new license is the Python Software Foundation License Version 2. Core and builtins ----------------- - Bug #1055820 Cyclic garbage collection was not protecting against that calling a live weakref to a piece of cyclic trash could resurrect an insane mutation of the trash if any Python code ran during gc (via running a dead object's __del__ method, running another callback on a weakref to a dead object, or via any Python code run in any other thread that managed to obtain the GIL while a __del__ or callback was running in the thread doing gc). The most likely symptom was "impossible" ``AttributeError`` exceptions, appearing seemingly at random, on weakly referenced objects. The cure was to clear all weakrefs to unreachable objects before allowing any callbacks to run. - Bug #1054139 _PyString_Resize() now invalidates its cached hash value. Extension Modules ----------------- - Bug #1048870: the compiler now generates distinct code objects for functions with identical bodies. This was producing confusing traceback messages which pointed to the function where the code object was first defined rather than the function being executed. Library ------- - Patch #1056967 changes the semantics of Template.safe_substitute() so that no ValueError is raised on an 'invalid' match group. Now the delimiter is returned. - Bug #1052503 pdb.runcall() was not passing along keyword arguments. - Bug #902037: XML.sax.saxutils.prepare_input_source() now combines relative paths with a base path before checking os.path.isfile(). - The whichdb module can now be run from the command line. - Bug #1045381: time.strptime() can now infer the date using %U or %W (week of the year) when the day of the week and year are also specified. - Bug #1048816: fix bug in Ctrl-K at start of line in curses.textpad.Textbox - Bug #1017553: fix bug in tarfile.filemode() - Patch #737473: fix bug that old source code is shown in tracebacks even if the source code is updated and reloaded. Build ----- - Patch #1044395: --enable-shared is allowed in FreeBSD also. What's New in Python 2.4 beta 1? ================================ *Release date: 15-OCT-2004* Core and builtins ----------------- - Patch #975056: Restartable signals were not correctly disabled on BSD systems. Consistently use PyOS_setsig() instead of signal(). - The internal portable implementation of thread-local storage (TLS), used by the ``PyGILState_Ensure()``/``PyGILState_Release()`` API, was not thread-correct. This could lead to a variety of problems, up to and including segfaults. See bug 1041645 for an example. - Added a command line option, -m module, which searches sys.path for the module and then runs it. (Contributed by Nick Coghlan.) - The bytecode optimizer now folds tuples of constants into a single constant. - SF bug #513866: Float/long comparison anomaly. Prior to 2.4b1, when an integer was compared to a float, the integer was coerced to a float. That could yield spurious overflow errors (if the integer was very large), and to anomalies such as ``long(1e200)+1 == 1e200 == long(1e200)-1``. Coercion to float is no longer performed, and cases like ``long(1e200)-1 < 1e200``, ``long(1e200)+1 > 1e200`` and ``(1 << 20000) > 1e200`` are computed correctly now. Extension modules ----------------- - ``collections.deque`` objects didn't play quite right with garbage collection, which could lead to a segfault in a release build, or an assert failure in a debug build. Also, added overflow checks, better detection of mutation during iteration, and shielded deque comparisons from unusual subclass overrides of the __iter__() method. Library ------- - Patch 1046644: distutils build_ext grew two new options - --swig for specifying the swig executable to use, and --swig-opts to specify options to pass to swig. --swig-opts="-c++" is the new way to spell --swig-cpp. - Patch 983206: distutils now obeys environment variable LDSHARED, if it is set. - Added Peter Astrand's subprocess.py module. See PEP 324 for details. - time.strptime() now properly escapes timezones and all other locale-specific strings for regex-specific symbols. Was breaking under Japanese Windows when the timezone was specified as "Tokyo (standard time)". Closes bug #1039270. - Updates for the email package: + email.Utils.formatdate() grew a 'usegmt' argument for HTTP support. + All deprecated APIs that in email 2.x issued warnings have been removed: _encoder argument to the MIMEText constructor, Message.add_payload(), Utils.dump_address_pair(), Utils.decode(), Utils.encode() + New deprecations: Generator.__call__(), Message.get_type(), Message.get_main_type(), Message.get_subtype(), the 'strict' argument to the Parser constructor. These will be removed in email 3.1. + Support for Python earlier than 2.3 has been removed (see PEP 291). + All defect classes have been renamed to end in 'Defect'. + Some FeedParser fixes; also a MultipartInvariantViolationDefect will be added to messages that claim to be multipart but really aren't. + Updates to documentation. - re's findall() and finditer() functions now take an optional flags argument just like the compile(), search(), and match() functions. Also, documented the previously existing start and stop parameters for the findall() and finditer() methods of regular expression objects. - rfc822 Messages now support iterating over the headers. - The (undocumented) tarfile.Tarfile.membernames has been removed; applications should use the getmember function. - httplib now offers symbolic constants for the HTTP status codes. - SF bug #1028306: Trying to compare a ``datetime.date`` to a ``datetime.datetime`` mistakenly compared only the year, month and day. Now it acts like a mixed-type comparison: ``False`` for ``==``, ``True`` for ``!=``, and raises ``TypeError`` for other comparison operators. Because datetime is a subclass of date, comparing only the base class (date) members can still be done, if that's desired, by forcing using of the approprate date method; e.g., ``a_date.__eq__(a_datetime)`` is true if and only if the year, month and day members of ``a_date`` and ``a_datetime`` are equal. - bdist_rpm now supports command line options --force-arch, {pre,post}-install, {pre,post}-uninstall, and {prep,build,install,clean,verify}-script. - SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support decoding incomplete input (when the input stream is temporarily exhausted). ``codecs.StreamReader`` now implements buffering, which enables proper readline support for the UTF-16 decoders. ``codecs.StreamReader.read()`` has a new argument ``chars`` which specifies the number of characters to return. ``codecs.StreamReader.readline()`` and ``codecs.StreamReader.readlines()`` have a new argument ``keepends``. Trailing "\n"s will be stripped from the lines if ``keepends`` is false. - The documentation for doctest is greatly expanded, and now covers all the new public features (of which there are many). - ``doctest.master`` was put back in, and ``doctest.testmod()`` once again updates it. This isn't good, because every ``testmod()`` call contributes to bloating the "hidden" state of ``doctest.master``, but some old code apparently relies on it. For now, all we can do is encourage people to stitch doctests together via doctest's unittest integration features instead. - httplib now handles ipv6 address/port pairs. - SF bug #1017864: ConfigParser now correctly handles default keys, processing them with ``ConfigParser.optionxform`` when supplied, consistent with the handling of config file entries and runtime-set options. - SF bug #997050: Document, test, & check for non-string values in ConfigParser. Moved the new string-only restriction added in rev. 1.65 to the SafeConfigParser class, leaving existing ConfigParser & RawConfigParser behavior alone, and documented the conditions under which non-string values work. Build ----- - Building on darwin now includes /opt/local/include and /opt/local/lib for building extension modules. This is so as to include software installed as a DarwinPorts port - pyport.h now defines a Py_IS_NAN macro. It works as-is when the platform C computes true for ``x != x`` if and only if X is a NaN. Other platforms can override the default definition with a platform- specific spelling in that platform's pyconfig.h. You can also override pyport.h's default Py_IS_INFINITY definition now. C API ----- - SF patch 1044089: New function ``PyEval_ThreadsInitialized()`` returns non-zero if PyEval_InitThreads() has been called. - The undocumented and unused extern int ``_PyThread_Started`` was removed. - The C API calls ``PyInterpreterState_New()`` and ``PyThreadState_New()`` are two of the very few advertised as being safe to call without holding the GIL. However, this wasn't true in a debug build, as bug 1041645 demonstrated. In a debug build, Python redirects the ``PyMem`` family of calls to Python's small-object allocator, to get the benefit of its extra debugging capabilities. But Python's small-object allocator isn't threadsafe, relying on the GIL to avoid the expense of doing its own locking. ``PyInterpreterState_New()`` and ``PyThreadState_New()`` call the platform ``malloc()`` directly now, regardless of build type. - PyLong_AsUnsignedLong[Mask] now support int objects as well. - SF patch #998993: ``PyUnicode_DecodeUTF8Stateful`` and ``PyUnicode_DecodeUTF16Stateful`` have been added, which implement stateful decoding. Tests ----- - test__locale ported to unittest Mac --- - ``plistlib`` now supports non-dict root objects. There is also a new interface for reading and writing plist files: ``readPlist(pathOrFile)`` and ``writePlist(rootObject, pathOrFile)`` Tools/Demos ----------- - The text file comparison scripts ``ndiff.py`` and ``diff.py`` now read the input files in universal-newline mode. This spares them from consuming a great deal of time to deduce the useless result that, e.g., a file with Windows line ends and a file with Linux line ends have no lines in common. What's New in Python 2.4 alpha 3? ================================= *Release date: 02-SEP-2004* Core and builtins ----------------- - SF patch #1007189: ``from ... import ...`` statements now allow the name list to be surrounded by parentheses. - Some speedups for long arithmetic, thanks to Trevor Perrin. Gradeschool multiplication was sped a little by optimizing the C code. Gradeschool squaring was sped by about a factor of 2, by exploiting that about half the digit products are duplicates in a square. Because exponentiation uses squaring often, this also speeds long power. For example, the time to compute 17**1000000 dropped from about 14 seconds to 9 on my box due to this much. The cutoff for Karatsuba multiplication was raised, since gradeschool multiplication got quicker, and the cutoff was aggressively small regardless. The exponentiation algorithm was switched from right-to-left to left-to-right, which is more efficient for small bases. In addition, if the exponent is large, the algorithm now does 5 bits (instead of 1 bit) at a time. That cut the time to compute 17**1000000 on my box in half again, down to about 4.5 seconds. - OverflowWarning is no longer generated. PEP 237 scheduled this to occur in Python 2.3, but since OverflowWarning was disabled by default, nobody realized it was still being generated. On the chance that user code is still using them, the Python builtin OverflowWarning, and corresponding C API PyExc_OverflowWarning, will exist until Python 2.5. - Py_InitializeEx has been added. - Fix the order of application of decorators. The proper order is bottom-up; the first decorator listed is the last one called. - SF patch #1005778. Fix a seg fault if the list size changed while calling list.index(). This could happen if a rich comparison function modified the list. - The ``func_name`` (a.k.a. ``__name__``) attribute of user-defined functions is now writable. - code_new (a.k.a new.code()) now checks its arguments sufficiently carefully that passing them on to PyCode_New() won't trigger calls to Py_FatalError() or PyErr_BadInternalCall(). It is still the case that the returned code object might be entirely insane. - Subclasses of string can no longer be interned. The semantics of interning were not clear here -- a subclass could be mutable, for example -- and had bugs. Explicitly interning a subclass of string via intern() will raise a TypeError. Internal operations that attempt to intern a string subclass will have no effect. - Bug 1003935: xrange() could report bogus OverflowErrors. Documented what xrange() intends, and repaired tests accordingly. Extension modules ----------------- - difflib now supports HTML side-by-side diff. - os.urandom has been added for systems that support sources of random data. - Patch 1012740: truncate() on a writeable cStringIO now resets the position to the end of the stream. This is consistent with the original StringIO module and avoids inadvertently resurrecting data that was supposed to have been truncated away. - Added socket.socketpair(). - Added CurrentByteIndex, CurrentColumnNumber, CurrentLineNumber members to xml.parsers.expat.XMLParser object. - The mpz, rotor, and xreadlines modules, all deprecated in earlier versions of Python, have now been removed. Library ------- - Patch #934356: if a module defines __all__, believe that rather than using heuristics for filtering out imported names. - Patch #941486: added os.path.lexists(), which returns True for broken symlinks, unlike os.path.exists(). - the random module now uses os.urandom() for seeding if it is available. Added a new generator based on os.urandom(). - difflib and diff.py can now generate HTML. - bdist_rpm now includes version and release in the BuildRoot, and replaces - by ``_`` in version and release. - distutils build/build_scripts now has an -e option to specify the path to the Python interpreter for installed scripts. - PEP 292 classes Template and SafeTemplate are added to the string module. - tarfile now generates GNU tar files by default. - HTTPResponse has now a getheaders method. - Patch #1006219: let inspect.getsource handle '@' decorators. Thanks Simon Percivall. - logging.handlers.SMTPHandler.date_time has been removed; the class now uses email.Utils.formatdate to generate the time stamp. - A new function tkFont.nametofont was added to return an existing font. The Font class constructor now has an additional exists argument which, if True, requests to return/configure an existing font, rather than creating a new one. - Updated the decimal package's min() and max() methods to match the latest revision of the General Decimal Arithmetic Specification. Quiet NaNs are ignored and equal values are sorted based on sign and exponent. - The decimal package's Context.copy() method now returns deep copies. - Deprecated sys.exitfunc in favor of the atexit module. The sys.exitfunc attribute will be kept around for backwards compatibility and atexit will just become the one preferred way to do it. - patch #675551: Add get_history_item and replace_history_item functions to the readline module. - bug #989672: pdb.doc and the help messages for the help_d and help_u methods of the pdb.Pdb class gives have been corrected. d(own) goes to a newer frame, u(p) to an older frame, not the other way around. - bug #990669: os.path.realpath() will resolve symlinks before normalizing the path, as normalizing the path may alter the meaning of the path if it contains symlinks. - bug #851123: shutil.copyfile will raise an exception when trying to copy a file onto a link to itself. Thanks Gregory Ball. - bug #570300: Fix inspect to resolve file locations using os.path.realpath() so as to properly list all functions in a module when the module itself is reached through a symlink. Thanks Johannes Gijsbers. - doctest refactoring continued. See the docs for details. As part of this effort, some old and little- (never?) used features are now deprecated: the Tester class, the module is_private() function, and the isprivate argument to testmod(). The Tester class supplied a feeble "by hand" way to combine multiple doctests, if you knew exactly what you were doing. The newer doctest features for unittest integration already did a better job of that, are stronger now than ever, and the new DocTestRunner class is a saner foundation if you want to do it by hand. The "private name" filtering gimmick was a mistake from the start, and testmod() changed long ago to ignore it by default. If you want to filter out tests, the new DocTestFinder class can be used to return a list of all doctests, and you can filter that list by any computable criteria before passing it to a DocTestRunner instance. - Bug #891637, patch #1005466: fix inspect.getargs() crash on def foo((bar)). Tools/Demos ----------- - IDLE's shortcut keys for windows are now case insensitive so that Control-V works the same as Control-v. - pygettext.py: Generate POT-Creation-Date header in ISO format. Build ----- - Backward incompatibility: longintrepr.h now triggers a compile-time error if SHIFT (the number of bits in a Python long "digit") isn't divisible by 5. This new requirement allows simple code for the new 5-bits-at-a-time long_pow() implementation. If necessary, the restriction could be removed (by complicating long_pow(), or by falling back to the 1-bit-at-a-time algorithm), but there are no plans to do so. - bug #991962: When building with --disable-toolbox-glue on Darwin no attempt to build Mac-specific modules occurs. - The --with-tsc flag to configure to enable VM profiling with the processor's timestamp counter now works on PPC platforms. - patch #1006629: Define _XOPEN_SOURCE to 500 on Solaris 8/9 to match GCC's definition and avoid redefinition warnings. - Detect pthreads support (provided by gnu pth pthread emulation) on GNU/k*BSD systems. - bug #1005737, #1007249: Fixed several build problems and warnings found on old/legacy C compilers of HP-UX, IRIX and Tru64. C API ----- .. Documentation ------------- - patch #1005936, bug #1009373: fix index entries which contain an underscore when viewed with Acrobat. - bug #990669: os.path.normpath may alter the meaning of a path if it contains symbolic links. This has been documented in a comment since 1992, but is now in the library reference as well. New platforms ------------- - FreeBSD 6 is now supported. Tests ----- .. Windows ------- - Boosted the stack reservation for python.exe and pythonw.exe from the default 1MB to 2MB. Stack frames under VC 7.1 for 2.4 are enough bigger than under VC 6.0 for 2.3.4 that deeply recursive progams within the default sys.getrecursionlimit() default value of 1000 were able to suffer undetected C stack overflows. The standard test program test_compiler was one such program. If a Python process on Windows "just vanishes" without a trace, and without an error message of any kind, but with an exit code of 128, undetected stack overflow may be the problem. Mac --- .. What's New in Python 2.4 alpha 2? ================================= *Release date: 05-AUG-2004* Core and builtins ----------------- - Patch #980695: Implements efficient string concatenation for statements of the form s=s+t and s+=t. This will vary across implementations. Accordingly, the str.join() method is strongly preferred for performance sensitive code. - PEP-0318, Function Decorators have been added to the language. These are implemented using the Java-style @decorator syntax, like so:: @staticmethod def foo(bar): (The PEP needs to be updated to reflect the current state) - When importing a module M raises an exception, Python no longer leaves M in sys.modules. Before 2.4a2 it did, and a subsequent import of M would succeed, picking up a module object from sys.modules reflecting as much of the initialization of M as completed before the exception was raised. Subsequent imports got no indication that M was in a partially- initialized state, and the importers could get into arbitrarily bad trouble as a result (the M they got was in an unintended state, arbitrarily far removed from M's author's intent). Now subsequent imports of M will continue raising exceptions (but if, for example, the source code for M is edited between import attempts, then perhaps later attempts will succeed, or raise a different exception). This can break existing code, but in such cases the code was probably working before by accident. In the Python source, the only case of breakage discovered was in a test accidentally relying on a damaged module remaining in sys.modules. Cases are also known where tests deliberately provoking import errors remove damaged modules from sys.modules themselves, and such tests will break now if they do an unconditional del sys.modules[M]. - u'%s' % obj will now try obj.__unicode__() first and fallback to obj.__str__() if no __unicode__ method can be found. - Patch #550732: Add PyArg_VaParseTupleAndKeywords(). Analogous to PyArg_VaParse(). Both are now documented. Thanks Greg Chapman. - Allow string and unicode return types from .encode()/.decode() methods on string and unicode objects. Added unicode.decode() which was missing for no apparent reason. - An attempt to fix the mess that is Python's behaviour with signal handlers and threads, complicated by readline's behaviour. It's quite possible that there are still bugs here. - Added C macros Py_CLEAR and Py_VISIT to ease the implementation of types that support garbage collection. - Compiler now treats None as a constant. - The type of values returned by __int__, __float__, __long__, __oct__, and __hex__ are now checked. Returning an invalid type will cause a TypeError to be raised. This matches the behavior of Jython. - Implemented bind_textdomain_codeset() in locale module. - Added a workaround for proper string operations in BSDs. str.split and str.is* methods can now work correctly with UTF-8 locales. - Bug #989185: unicode.iswide() and unicode.width() is dropped and the East Asian Width support is moved to unicodedata extension module. - Patch #941229: The source code encoding in interactive mode now refers sys.stdin.encoding not just ISO-8859-1 anymore. This allows for non-latin-1 users to write unicode strings directly. Extension modules ----------------- - cpickle now supports the same keyword arguments as pickle. Library ------- - Added new codecs and aliases for ISO_8859-11, ISO_8859-16 and TIS-620 - Thanks to Edward Loper, doctest has been massively refactored, and many new features were added. Full docs will appear later. For now the doctest module comments and new test cases give good coverage. The refactoring provides many hook points for customizing behavior (such as how to report errors, and how to compare expected to actual output). New features include a marker for expected output containing blank lines, options to produce unified or context diffs when actual output doesn't match expectations, an option to normalize whitespace before comparing, and an option to use an ellipsis to signify "don't care" regions of output. - Tkinter now supports the wish -sync and -use options. - The following methods in time support passing of None: ctime(), gmtime(), and localtime(). If None is provided, the current time is used (the same as when the argument is omitted). [SF bug 658254, patch 663482] - nntplib does now allow to ignore a .netrc file. - urllib2 now recognizes Basic authentication even if other authentication schemes are offered. - Bug #1001053. wave.open() now accepts unicode filenames. - gzip.GzipFile has a new fileno() method, to retrieve the handle of the underlying file object (provided it has a fileno() method). This is needed if you want to use os.fsync() on a GzipFile. - imaplib has two new methods: deleteacl and myrights. - nntplib has two new methods: description and descriptions. They use a more RFC-compliant way of getting a newsgroup description. - Bug #993394. Fix a possible red herring of KeyError in 'threading' being raised during interpreter shutdown from a registered function with atexit when dummy_threading is being used. - Bug #857297/Patch #916874. Fix an error when extracting a hard link from a tarfile. - Patch #846659. Fix an error in tarfile.py when using GNU longname/longlink creation. - The obsolete FCNTL.py has been deleted. The builtin fcntl module has been available (on platforms that support fcntl) since Python 1.5a3, and all FCNTL.py did is export fcntl's names, after generating a deprecation warning telling you to use fcntl directly. - Several new unicode codecs are added: big5hkscs, euc_jis_2004, iso2022_jp_2004, shift_jis_2004. - Bug #788520. Queue.{get, get_nowait, put, put_nowait} have new implementations, exploiting Conditions (which didn't exist at the time Queue was introduced). A minor semantic change is that the Full and Empty exceptions raised by non-blocking calls now occur only if the queue truly was full or empty at the instant the queue was checked (of course the Queue may no longer be full or empty by the time a calling thread sees those exceptions, though). Before, the exceptions could also be raised if it was "merely inconvenient" for the implementation to determine the true state of the Queue (because the Queue was locked by some other method in progress). - Bugs #979794 and #980117: difflib.get_grouped_opcodes() now handles the case of comparing two empty lists. This affected both context_diff() and unified_diff(), - Bug #980938: smtplib now prints debug output to sys.stderr. - Bug #930024: posixpath.realpath() now handles infinite loops in symlinks by returning the last point in the path that was not part of any loop. Thanks AM Kuchling. - Bug #980327: ntpath not handles compressing erroneous slashes between the drive letter and the rest of the path. Also clearly handles UNC addresses now as well. Thanks Paul Moore. - bug #679953: zipfile.py should now work for files over 2 GB. The packed data for file sizes (compressed and uncompressed) was being stored as signed instead of unsigned. - decimal.py now only uses signals in the IBM spec. The other conditions are no longer part of the public API. - codecs module now has two new generic APIs: encode() and decode() which don't restrict the return types (unlike the unicode and string methods of the same name). - Non-blocking SSL sockets work again; they were broken in Python 2.3. SF patch 945642. - doctest unittest integration improvements: o Improved the unitest test output for doctest-based unit tests o Can now pass setUp and tearDown functions when creating DocTestSuites. - The threading module has a new class, local, for creating objects that provide thread-local data. - Bug #990307: when keep_empty_values is True, cgi.parse_qsl() no longer returns spurious empty fields. - Implemented bind_textdomain_codeset() in gettext module. - Introduced in gettext module the l*gettext() family of functions, which return translation strings encoded in the preferred encoding, as informed by locale module's getpreferredencoding(). - optparse module (and tests) upgraded to Optik 1.5a1. Changes: - Add expansion of default values in help text: the string "%default" in an option's help string is expanded to str() of that option's default value, or "none" if no default value. - Bug #955889: option default values that happen to be strings are now processed in the same way as values from the command line; this allows generation of nicer help when using custom types. Can be disabled with parser.set_process_default_values(False). - Bug #960515: don't crash when generating help for callback options that specify 'type', but not 'dest' or 'metavar'. - Feature #815264: change the default help format for short options that take an argument from e.g. "-oARG" to "-o ARG"; add set_short_opt_delimiter() and set_long_opt_delimiter() methods to HelpFormatter to allow (slight) customization of the formatting. - Patch #736940: internationalize Optik: all built-in user- targeted literal strings are passed through gettext.gettext(). (If you want translations (.po files), they're not included with Python -- you'll find them in the Optik source distribution from http://optik.sourceforge.net/ .) - Bug #878453: respect $COLUMNS environment variable for wrapping help output. - Feature #988122: expand "%prog" in the 'description' passed to OptionParser, just like in the 'usage' and 'version' strings. (This is *not* done in the 'description' passed to OptionGroup.) C API ----- - PyImport_ExecCodeModule() and PyImport_ExecCodeModuleEx(): if an error occurs while loading the module, these now delete the module's entry from sys.modules. All ways of loading modules eventually call one of these, so this is an error-case change in semantics for all ways of loading modules. In rare cases, a module loader may wish to keep a module object in sys.modules despite that the module's code cannot be executed. In such cases, the module loader must arrange to reinsert the name and module object in sys.modules. PyImport_ReloadModule() has been changed to reinsert the original module object into sys.modules if the module reload fails, so that its visible semantics have not changed. - A large pile of datetime field-extraction macros is now documented, thanks to Anthony Tuininga (patch #986010). Documentation ------------- - Improved the tutorial on creating types in C. - point out the importance of reassigning data members before assigning their values - correct my misconception about return values from visitprocs. Sigh. - mention the labor saving Py_VISIT and Py_CLEAR macros. - Major rewrite of the math module docs, to address common confusions. Tests ----- - The test data files for the decimal test suite are now installed on platforms that use the Makefile. - SF patch 995225: The test file testtar.tar accidentally contained CVS keywords (like $Id$), which could cause spurious failures in test_tarfile.py depending on how the test file was checked out. What's New in Python 2.4 alpha 1? ================================= *Release date: 08-JUL-2004* Core and builtins ----------------- - weakref.ref is now the type object also known as weakref.ReferenceType; it can be subclassed like any other new-style class. There's less per-entry overhead in WeakValueDictionary objects now (one object instead of three). - Bug #951851: Python crashed when reading import table of certain Windows DLLs. - Bug #215126. The locals argument to eval(), execfile(), and exec now accept any mapping type. - marshal now shares interned strings. This change introduces a new .pyc magic. - Bug #966623. classes created with type() in an exec(, {}) don't have a __module__, but code in typeobject assumed it would always be there. - Python no longer relies on the LC_NUMERIC locale setting to be the "C" locale; as a result, it no longer tries to prevent changing the LC_NUMERIC category. - Bug #952807: Unpickling pickled instances of subclasses of datetime.date, datetime.datetime and datetime.time could yield insane objects. Thanks to Jiwon Seo for a fix. - Bug #845802: Python crashes when __init__.py is a directory. - Unicode objects received two new methods: iswide() and width(). These query East Asian width information, as specified in Unicode TR11. - Improved the tuple hashing algorithm to give fewer collisions in common cases. Fixes bug #942952. - Implemented generator expressions (PEP 289). Coded by Jiwon Seo. - Enabled the profiling of C extension functions (and builtins) - check new documentation and modified profile and bdb modules for more details - Set file.name to the object passed to open (instead of a new string) - Moved tracebackobject into traceback.h and renamed to PyTracebackObject - Optimized the byte coding for multiple assignments like "a,b=b,a" and "a,b,c=1,2,3". Improves their speed by 25% to 30%. - Limit the nested depth of a tuple for the second argument to isinstance() and issubclass() to the recursion limit of the interpreter. Fixes bug #858016 . - Optimized dict iterators, creating separate types for each and having them reveal their length. Also optimized the methods: keys(), values(), and items(). - Implemented a newcode opcode, LIST_APPEND, that simplifies the generated bytecode for list comprehensions and further improves their performance (about 35%). - Implemented rich comparisons for floats, which seems to make comparisons involving NaNs somewhat less surprising when the underlying C compiler actually implements C99 semantics. - Optimized list.extend() to save memory and no longer create intermediate sequences. Also, extend() now pre-allocates the needed memory whenever the length of the iterable is known in advance -- this halves the time to extend the list. - Optimized list resize operations to make fewer calls to the system realloc(). Significantly speeds up list appends, list pops, list comprehensions, and the list constructor (when the input iterable length is not known). - Changed the internal list over-allocation scheme. For larger lists, overallocation ranged between 3% and 25%. Now, it is a constant 12%. For smaller lists (n<8), overallocation was upto eight elements. Now, the overallocation is no more than three elements -- this improves space utilization for applications that have large numbers of small lists. - Most list bodies now get re-used rather than freed. Speeds up list instantiation and deletion by saving calls to malloc() and free(). - The dict.update() method now accepts all the same argument forms as the dict() constructor. This now includes item lists and/or keyword arguments. - Support for arbitrary objects supporting the read-only buffer interface as the co_code field of code objects (something that was only possible to create from C code) has been removed. - Made omitted callback and None equivalent for weakref.ref() and weakref.proxy(); the None case wasn't handled correctly in all cases. - Fixed problem where PyWeakref_NewRef() and PyWeakref_NewProxy() assumed that initial existing entries in an object's weakref list would not be removed while allocating a new weakref object. Since GC could be invoked at that time, however, that assumption was invalid. In a truly obscure case of GC being triggered during creation for a new weakref object for an referent which already has a weakref without a callback which is only referenced from cyclic trash, a memory error can occur. This consistently created a segfault in a debug build, but provided less predictable behavior in a release build. - input() builtin function now respects compiler flags such as __future__ statements. SF patch 876178. - Removed PendingDeprecationWarning from apply(). apply() remains deprecated, but the nuisance warning will not be issued. - At Python shutdown time (Py_Finalize()), 2.3 called cyclic garbage collection twice, both before and after tearing down modules. The call after tearing down modules has been disabled, because too much of Python has been torn down then for __del__ methods and weakref callbacks to execute sanely. The most common symptom was a sequence of uninformative messages on stderr when Python shut down, produced by threads trying to raise exceptions, but unable to report the nature of their problems because too much of the sys module had already been destroyed. - Removed FutureWarnings related to hex/oct literals and conversions and left shifts. (Thanks to Kalle Svensson for SF patch 849227.) This addresses most of the remaining semantic changes promised by PEP 237, except for repr() of a long, which still shows the trailing 'L'. The PEP appears to promise warnings for operations that changed semantics compared to Python 2.3, but this is not implemented; we've suffered through enough warnings related to hex/oct literals and I think it's best to be silent now. - For str and unicode objects, the ljust(), center(), and rjust() methods now accept an optional argument specifying a fill character other than a space. - When method objects have an attribute that can be satisfied either by the function object or by the method object, the function object's attribute usually wins. Christian Tismer pointed out that that this is really a mistake, because this only happens for special methods (like __reduce__) where the method object's version is really more appropriate than the function's attribute. So from now on, all method attributes will have precedence over function attributes with the same name. - Critical bugfix, for SF bug 839548: if a weakref with a callback, its callback, and its weakly referenced object, all became part of cyclic garbage during a single run of garbage collection, the order in which they were torn down was unpredictable. It was possible for the callback to see partially-torn-down objects, leading to immediate segfaults, or, if the callback resurrected garbage objects, to resurrect insane objects that caused segfaults (or other surprises) later. In one sense this wasn't surprising, because Python's cyclic gc had no knowledge of Python's weakref objects. It does now. When weakrefs with callbacks become part of cyclic garbage now, those weakrefs are cleared first. The callbacks don't trigger then, preventing the problems. If you need callbacks to trigger, then just as when cyclic gc is not involved, you need to write your code so that weakref objects outlive the objects they weakly reference. - Critical bugfix, for SF bug 840829: if cyclic garbage collection happened to occur during a weakref callback for a new-style class instance, subtle memory corruption was the result (in a release build; in a debug build, a segfault occurred reliably very soon after). This has been repaired. - Compiler flags set in PYTHONSTARTUP are now active in __main__. - Added two builtin types, set() and frozenset(). - Added a reversed() builtin function that returns a reverse iterator over a sequence. - Added a sorted() builtin function that returns a new sorted list from any iterable. - CObjects are now mutable (on the C level) through PyCObject_SetVoidPtr. - list.sort() now supports three keyword arguments: cmp, key, and reverse. The key argument can be a function of one argument that extracts a comparison key from the original record: mylist.sort(key=str.lower). The reverse argument is a boolean value and if True will change the sort order as if the comparison arguments were reversed. In addition, the documentation has been amended to provide a guarantee that all sorts starting with Py2.3 are guaranteed to be stable (the relative order of records with equal keys is unchanged). - Added test whether wchar_t is signed or not. A signed wchar_t is not usable as internal unicode type base for Py_UNICODE since the unicode implementation assumes an unsigned type. - Fixed a bug in the cache of length-one Unicode strings that could lead to a seg fault. The specific problem occurred when an earlier, non-fatal error left an uninitialized Unicode object in the freelist. - The % formatting operator now supports '%F' which is equivalent to '%f'. This has always been documented but never implemented. - complex(obj) could leak a little memory if obj wasn't a string or number. - zip() with no arguments now returns an empty list instead of raising a TypeError exception. - obj.__contains__() now returns True/False instead of 1/0. SF patch 820195. - Python no longer tries to be smart about recursive comparisons. When comparing containers with cyclic references to themselves it will now just hit the recursion limit. See SF patch 825639. - str and unicode builtin types now have an rsplit() method that is same as split() except that it scans the string from the end working towards the beginning. See SF feature request 801847. - Fixed a bug in object.__reduce_ex__ when using protocol 2. Failure to clear the error when attempts to get the __getstate__ attribute fail caused intermittent errors and odd behavior. - buffer objects based on other objects no longer cache a pointer to the data and the data length. Instead, the appropriate tp_as_buffer method is called as necessary. - fixed: if a file is opened with an explicit buffer size >= 1, repeated close() calls would attempt to free() the buffer already free()ed on the first call. Extension modules ----------------- - Added socket.getservbyport(), and make the second argument in getservbyname() and getservbyport() optional. - time module code that deals with input POSIX timestamps will now raise ValueError if more than a second is lost in precision when the timestamp is cast to the platform C time_t type. There's no chance that the platform will do anything sensible with the result in such cases. This includes ctime(), localtime() and gmtime(). Assorted fromtimestamp() and utcfromtimestamp() methods in the datetime module were also protected. Closes bugs #919012 and 975996. - fcntl.ioctl now warns if the mutate flag is not specified. - nt now properly allows to refer to UNC roots, e.g. in nt.stat(). - the weakref module now supports additional objects: array.array, sre.pattern_objects, file objects, and sockets. - operator.isMappingType() and operator.isSequenceType() now give fewer false positives. - socket.sslerror is now a subclass of socket.error . Also added socket.error to the socket module's C API. - Bug #920575: A problem where the _locale module segfaults on nl_langinfo(ERA) caused by GNU libc's illegal NULL return is fixed. - array objects now support the copy module. Also, their resizing scheme has been updated to match that used for list objects. This improves the performance (speed and memory usage) of append() operations. Also, array.array() and array.extend() now accept any iterable argument for repeated appends without needing to create another temporary array. - cStringIO.writelines() now accepts any iterable argument and writes the lines one at a time rather than joining them and writing once. Made a parallel change to StringIO.writelines(). Saves memory and makes suitable for use with generator expressions. - time.strftime() now checks that the values in its time tuple argument are within the proper boundaries to prevent possible crashes from the platform's C library implementation of strftime(). Can possibly break code that uses values outside the range that didn't cause problems previously (such as sitting day of year to 0). Fixes bug #897625. - The socket module now supports Bluetooth sockets, if the system has - Added a collections module containing a new datatype, deque(), offering high-performance, thread-safe, memory friendly appends and pops on either side of the deque. - Several modules now take advantage of collections.deque() for improved performance: Queue, mutex, shlex, threading, and pydoc. - The operator module has two new functions, attrgetter() and itemgetter() which are useful for creating fast data extractor functions for map(), list.sort(), itertools.groupby(), and other functions that expect a function argument. - socket.SHUT_{RD,WR,RDWR} was added. - os.getsid was added. - The pwd module incorrectly advertised its struct type as struct_pwent; this has been renamed to struct_passwd. (The old name is still supported for backwards compatibility.) - The xml.parsers.expat module now provides Expat 1.95.7. - socket.IPPROTO_IPV6 was added. - readline.clear_history was added. - select.select() now accepts sequences for its first three arguments. - cStringIO now supports the f.closed attribute. - The signal module now exposes SIGRTMIN and SIGRTMAX (if available). - curses module now supports use_default_colors(). [patch #739124] - Bug #811028: ncurses.h breakage on FreeBSD/MacOS X - Bug #814613: INET_ADDRSTRLEN fix needed for all compilers on SGI - Implemented non-r