Merge branch 'main' into multiprocessing/default-start-method-3.14

.github/workflows/jit.yml

@@ -110,17 +110,15 @@ jobs:
       - name: Native Windows
         if: runner.os == 'Windows' && matrix.architecture != 'ARM64'
         run: |
-          choco upgrade llvm -y
-          choco install llvm --allow-downgrade --no-progress --version ${{ matrix.llvm }}
+          choco install llvm --allow-downgrade --no-progress --version ${{ matrix.llvm }}.1.0
           ./PCbuild/build.bat --experimental-jit ${{ matrix.debug && '-d' || '--pgo' }} -p ${{ matrix.architecture }}
           ./PCbuild/rt.bat ${{ matrix.debug && '-d' || '' }} -p ${{ matrix.architecture }} -q --multiprocess 0 --timeout 4500 --verbose2 --verbose3
 
       # No PGO or tests (yet):
       - name: Emulated Windows
         if: runner.os == 'Windows' && matrix.architecture == 'ARM64'
         run: |
-          choco upgrade llvm -y
-          choco install llvm --allow-downgrade --no-progress --version ${{ matrix.llvm }}
+          choco install llvm --allow-downgrade --no-progress --version ${{ matrix.llvm }}.1.0
           ./PCbuild/build.bat --experimental-jit ${{ matrix.debug && '-d' || '' }} -p ${{ matrix.architecture }}
 
       - name: Native macOS

Doc/c-api/contextvars.rst

@@ -101,6 +101,52 @@ Context object management functions:
    current context for the current thread.  Returns ``0`` on success,
    and ``-1`` on error.
 
+.. c:function:: int PyContext_AddWatcher(PyContext_WatchCallback callback)
+
+   Register *callback* as a context object watcher for the current interpreter.
+   Return an ID which may be passed to :c:func:`PyContext_ClearWatcher`.
+   In case of error (e.g. no more watcher IDs available),
+   return ``-1`` and set an exception.
+
+   .. versionadded:: 3.14
+
+.. c:function:: int PyContext_ClearWatcher(int watcher_id)
+
+   Clear watcher identified by *watcher_id* previously returned from
+   :c:func:`PyContext_AddWatcher` for the current interpreter.
+   Return ``0`` on success, or ``-1`` and set an exception on error
+   (e.g. if the given *watcher_id* was never registered.)
+
+   .. versionadded:: 3.14
+
+.. c:type:: PyContextEvent
+
+   Enumeration of possible context object watcher events:
+   - ``Py_CONTEXT_EVENT_ENTER``
+   - ``Py_CONTEXT_EVENT_EXIT``
+
+   .. versionadded:: 3.14
+
+.. c:type:: int (*PyContext_WatchCallback)(PyContextEvent event, PyContext* ctx)
+
+   Type of a context object watcher callback function.
+   If *event* is ``Py_CONTEXT_EVENT_ENTER``, then the callback is invoked
+   after *ctx* has been set as the current context for the current thread.
+   Otherwise, the callback is invoked before the deactivation of *ctx* as the current context
+   and the restoration of the previous contex object for the current thread.
+
+   If the callback returns with an exception set, it must return ``-1``; this
+   exception will be printed as an unraisable exception using
+   :c:func:`PyErr_FormatUnraisable`. Otherwise it should return ``0``.
+
+   There may already be a pending exception set on entry to the callback. In
+   this case, the callback should return ``0`` with the same exception still
+   set. This means the callback may not call any other API that can set an
+   exception unless it saves and clears the exception state first, and restores
+   it before returning.
+
+   .. versionadded:: 3.14
+
 
 Context variable functions:
 

Doc/library/argparse.rst

@@ -249,11 +249,12 @@ The following sections describe how each of these are used.
 prog
 ^^^^
 
-By default, :class:`ArgumentParser` objects use ``sys.argv[0]`` to determine
+By default, :class:`ArgumentParser` objects use the base name
+(see :func:`os.path.basename`) of ``sys.argv[0]`` to determine
 how to display the name of the program in help messages.  This default is almost
-always desirable because it will make the help messages match how the program was
-invoked on the command line.  For example, consider a file named
-``myprogram.py`` with the following code::
+always desirable because it will make the help messages match the name that was
+used to invoke the program on the command line.  For example, consider a file
+named ``myprogram.py`` with the following code::
 
    import argparse
    parser = argparse.ArgumentParser()
@@ -1122,6 +1123,9 @@ is used when no command-line argument was present::
    >>> parser.parse_args([])
    Namespace(foo=42)
 
+For required_ arguments, the ``default`` value is ignored. For example, this
+applies to positional arguments with nargs_ values other than ``?`` or ``*``,
+or optional arguments marked as ``required=True``.
 
 Providing ``default=argparse.SUPPRESS`` causes no attribute to be added if the
 command-line argument was not present::

Doc/library/pdb.rst

@@ -159,12 +159,15 @@ slightly different way:
    is entered.
 
 
-.. function:: set_trace(*, header=None)
+.. function:: set_trace(*, header=None, commands=None)
 
    Enter the debugger at the calling stack frame.  This is useful to hard-code
    a breakpoint at a given point in a program, even if the code is not
    otherwise being debugged (e.g. when an assertion fails).  If given,
    *header* is printed to the console just before debugging begins.
+   The *commands* argument, if given, is a list of commands to execute
+   when the debugger starts.
+
 
    .. versionchanged:: 3.7
       The keyword-only argument *header*.
@@ -173,6 +176,9 @@ slightly different way:
       :func:`set_trace` will enter the debugger immediately, rather than
       on the next line of code to be executed.
 
+   .. versionadded:: 3.14
+      The *commands* argument.
+
 .. function:: post_mortem(traceback=None)
 
    Enter post-mortem debugging of the given *traceback* object.  If no

Doc/library/typing.rst

@@ -1458,6 +1458,23 @@ These can be used as types in annotations. They all support subscription using
         >>> X.__metadata__
         ('very', 'important', 'metadata')
 
+   * At runtime, if you want to retrieve the original
+     type wrapped by ``Annotated``, use the :attr:`!__origin__` attribute:
+
+     .. doctest::
+
+        >>> from typing import Annotated, get_origin
+        >>> Password = Annotated[str, "secret"]
+        >>> Password.__origin__
+        <class 'str'>
+
+     Note that using :func:`get_origin` will return ``Annotated`` itself:
+
+     .. doctest::
+
+        >>> get_origin(Password)
+        typing.Annotated
+
    .. seealso::
 
       :pep:`593` - Flexible function and variable annotations
@@ -3298,6 +3315,7 @@ Introspection helpers
       assert get_origin(str) is None
       assert get_origin(Dict[str, int]) is dict
       assert get_origin(Union[int, str]) is Union
+      assert get_origin(Annotated[str, "metadata"]) is Annotated
       P = ParamSpec('P')
       assert get_origin(P.args) is P
       assert get_origin(P.kwargs) is P

Doc/using/configure.rst

@@ -183,12 +183,6 @@ General Options
 
    See :envvar:`PYTHONCOERCECLOCALE` and the :pep:`538`.
 
-.. option:: --without-freelists
-
-   Disable all freelists except the empty tuple singleton.
-
-   .. versionadded:: 3.11
-
 .. option:: --with-platlibdir=DIRNAME
 
    Python library directory name (default is ``lib``).

Doc/whatsnew/3.11.rst

@@ -2168,7 +2168,7 @@ Build Changes
   (Contributed by Donghee Na and Brett Holman in :issue:`44340`.)
 
 * Freelists for object structs can now be disabled. A new :program:`configure`
-  option :option:`--without-freelists` can be used to disable all freelists
+  option ``--without-freelists`` can be used to disable all freelists
   except empty tuple singleton.
   (Contributed by Christian Heimes in :issue:`45522`.)
 

Doc/whatsnew/3.14.rst

@@ -70,6 +70,91 @@ Summary -- Release highlights
 New Features
 ============
 
+.. _whatsnew-314-pep649:
+
+PEP 649: Deferred Evaluation of Annotations
+-------------------------------------------
+
+The :term:`annotations <annotation>` on functions, classes, and modules are no
+longer evaluated eagerly. Instead, annotations are stored in special-purpose
+:term:`annotate functions <annotate function>` and evaluated only when
+necessary. This is specified in :pep:`649` and :pep:`749`.
+
+This change is designed to make annotations in Python more performant and more
+usable in most circumstances. The runtime cost for defining annotations is
+minimized, but it remains possible to introspect annotations at runtime.
+It is usually no longer necessary to enclose annotations in strings if they
+contain forward references.
+
+The new :mod:`annotationlib` module provides tools for inspecting deferred
+annotations. Annotations may be evaluated in the :attr:`~annotationlib.Format.VALUE`
+format (which evaluates annotations to runtime values, similar to the behavior in
+earlier Python versions), the :attr:`~annotationlib.Format.FORWARDREF` format
+(which replaces undefined names with special markers), and the
+:attr:`~annotationlib.Format.SOURCE` format (which returns annotations as strings).
+
+This example shows how these formats behave:
+
+.. doctest::
+
+   >>> from annotationlib import get_annotations, Format
+   >>> def func(arg: Undefined):
+   ...     pass
+   >>> get_annotations(func, format=Format.VALUE)
+   Traceback (most recent call last):
+     ...
+   NameError: name 'Undefined' is not defined
+   >>> get_annotations(func, format=Format.FORWARDREF)
+   {'arg': ForwardRef('Undefined')}
+   >>> get_annotations(func, format=Format.SOURCE)
+   {'arg': 'Undefined'}
+
+Implications for annotated code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you define annotations in your code (for example, for use with a static type
+checker), then this change probably does not affect you: you can keep
+writing annotations the same way you did with previous versions of Python.
+
+You will likely be able to remove quoted strings in annotations, which are frequently
+used for forward references. Similarly, if you use ``from __future__ import annotations``
+to avoid having to write strings in annotations, you may well be able to
+remove that import. However, if you rely on third-party libraries that read annotations,
+those libraries may need changes to support unquoted annotations before they
+work as expected.
+
+Implications for readers of ``__annotations__``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If your code reads the ``__annotations__`` attribute on objects, you may want
+to make changes in order to support code that relies on deferred evaluation of
+annotations. For example, you may want to use :func:`annotationlib.get_annotations`
+with the :attr:`~annotationlib.Format.FORWARDREF` format, as the :mod:`dataclasses`
+module now does.
+
+Related changes
+^^^^^^^^^^^^^^^
+
+The changes in Python 3.14 are designed to rework how ``__annotations__``
+works at runtime while minimizing breakage to code that contains
+annotations in source code and to code that reads ``__annotations__``. However,
+if you rely on undocumented details of the annotation behavior or on private
+functions in the standard library, there are many ways in which your code may
+not work in Python 3.14. To safeguard your code against future changes,
+use only the documented functionality of the :mod:`annotationlib` module.
+
+``from __future__ import annotations``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In Python 3.7, :pep:`563` introduced the ``from __future__ import annotations``
+directive, which turns all annotations into strings. This directive is now
+considered deprecated and it is expected to be removed in a future version of Python.
+However, this removal will not happen until after Python 3.13, the last version of
+Python without deferred evaluation of annotations, reaches its end of life.
+In Python 3.14, the behavior of code using ``from __future__ import annotations``
+is unchanged.
+
+
 Improved Error Messages
 -----------------------
 
@@ -109,7 +194,9 @@ Other Language Changes
 New Modules
 ===========
 
-* None yet.
+* :mod:`annotationlib`: For introspecting :term:`annotations <annotation>`.
+  See :pep:`749` for more details.
+  (Contributed by Jelle Zijlstra in :gh:`119180`.)
 
 
 Improved Modules
@@ -563,9 +650,10 @@ New Features
 Porting to Python 3.14
 ----------------------
 
-* In the limited C API 3.14 and newer, :c:func:`Py_TYPE` is now implemented as
-  an opaque function call to hide implementation details.
-  (Contributed by Victor Stinner in :gh:`120600`.)
+* In the limited C API 3.14 and newer, :c:func:`Py_TYPE` and
+  :c:func:`Py_REFCNT` are now implemented as an opaque function call to hide
+  implementation details.
+  (Contributed by Victor Stinner in :gh:`120600` and :gh:`124127`.)
 
 
 Deprecated

Include/cpython/context.h

@@ -27,6 +27,38 @@ PyAPI_FUNC(PyObject *) PyContext_CopyCurrent(void);
 PyAPI_FUNC(int) PyContext_Enter(PyObject *);
 PyAPI_FUNC(int) PyContext_Exit(PyObject *);
 
+typedef enum {
+   Py_CONTEXT_EVENT_ENTER,
+   Py_CONTEXT_EVENT_EXIT,
+} PyContextEvent;
+
+/*
+ * A Callback to clue in non-python contexts impls about a
+ * change in the active python context.
+ *
+ * The callback is invoked with the event and a reference to =
+ * the context after its entered and before its exited.
+ *
+ * if the callback returns with an exception set, it must return -1. Otherwise
+ * it should return 0
+ */
+typedef int (*PyContext_WatchCallback)(PyContextEvent, PyContext *);
+
+/*
+ * Register a per-interpreter callback that will be invoked for context object
+ * enter/exit events.
+ *
+ * Returns a handle that may be passed to PyContext_ClearWatcher on success,
+ * or -1 and sets and error if no more handles are available.
+ */
+PyAPI_FUNC(int) PyContext_AddWatcher(PyContext_WatchCallback callback);
+
+/*
+ * Clear the watcher associated with the watcher_id handle.
+ *
+ * Returns 0 on success or -1 if no watcher exists for the provided id.
+ */
+PyAPI_FUNC(int) PyContext_ClearWatcher(int watcher_id);
 
 /* Create a new context variable.
 

Include/internal/pycore_context.h

@@ -7,6 +7,7 @@
 
 #include "pycore_hamt.h"          // PyHamtObject
 
+#define CONTEXT_MAX_WATCHERS 8
 
 extern PyTypeObject _PyContextTokenMissing_Type;