diff --git a/peps/pep-0823.rst b/peps/pep-0823.rst index 1ba141a952d..1381d613e67 100644 --- a/peps/pep-0823.rst +++ b/peps/pep-0823.rst @@ -168,7 +168,7 @@ The ``?.`` and ``?[ ]`` operators can also aid in the traversal of structured data, oftentimes coming from JSON and parsed as nested dicts and lists. It is worth noting though that the operators do **NOT** handle ``missing`` attributes / data. In these cases, at -least for dictionaries a useful helper method is ``dict.get(key)``. +least for dictionaries a useful helper method is :py:meth:`dict.get`. .. code-block:: python @@ -435,16 +435,15 @@ Assignments are not permitted and will raise a ``SyntaxError``. This does not apply if the ``None``-aware expressions is only part of a larger expression and evaluated on its own, for example as a -function argument. +function argument or subscript. .. code-block:: py823-console - >>> a = None - >>> def f(a): - ... return [a, 0] - ... - - >>> f(a?.b)[1] = 1 + >>> a = {} + >>> b = None + >>> a[b?.c] = 2 + >>> a + {None: 2} Await expressions ***************** @@ -613,7 +612,7 @@ value other than ``None`` are not suppressed: * Similarly, if the key or index in ``a?[b]`` does not exist, you get a ``KeyError`` or ``IndexError``. To turn a missing key into ``None``, assuming ``a`` is a dictionary, you can use ``a?.get(b)``. There is no - equivalent of ``dict.get`` for lists. + equivalent of :py:meth:`dict.get` for lists. The expression may still be ``None`` when ``a`` is not ``None``. For example, ``a?.b`` returns ``None`` if ``a`` is not ``None`` but ``a.b`` is @@ -688,7 +687,8 @@ Coalesce ``??`` and coalesce assignment operator ``??=`` ``??`` and a "``None`` coalescing assignment" operator ``??=``. As the ``None``-aware access operators have their own use cases, the coalescing operators were moved into a separate document, see :pep:`824`. -Both proposals can be adopted independently of each other. +Both proposals can be adopted independently of each other, though they +do work well together. ``None``-aware function calls ----------------------------- @@ -728,8 +728,8 @@ Add ``list.get(key)`` --------------------- It was suggested to add a ``.get(key)`` method to ``list`` -and ``tuple`` objects, similar to the existing ``dict.get`` method. This -could further make parsing of structured data easier since it would no +and ``tuple`` objects, similar to the existing :py:meth:`dict.get` method. +This could further make parsing of structured data easier since it would no longer be necessary to check if a ``list`` or ``tuple`` is long enough before trying to access the n-th element avoiding a possible ``IndexError``. While potentially useful, the idea is out of the scope @@ -753,8 +753,8 @@ While this would technically work, it's not at all clear what the result should be if an error is caught. Furthermore, this approach would hide genuine issues like a misspelled attribute which would have raised an ``AttributeError``. There are also already established patterns to -handle these kinds of errors in the form of ``getattr`` and -``dict.get(key)``. +handle these kinds of errors in the form of :py:func:`getattr` and +:py:meth:`dict.get`. As catching exceptions would be unexpected and hide potential errors, it is rejected. @@ -847,6 +847,31 @@ If future PEPs want to introduce new operators to access attributes or call methods, e.g. a chaining operator, it would be advisable to consider if a ``None``-aware variant for it could be useful, at that time. +``None``-aware binary operators +------------------------------- + +Some suggested to add ``None``-aware binary operators, like ``?+``, +``?*``, ``?&``. Similar to the `previous `_ +section, it is not clear what a ``None``-aware binary operator would +mean. Both ``?.`` and ``?[ ]`` are access, not binary operators. + +Add ``?`` as shortcut for ``is not None`` +----------------------------------------- + +It was proposed that ``?`` could be interpreted as a shortcut for +``is not None``. + +.. code-block:: py823 + + # _t.c.d if ((_t := a.b) is not None) else None + _t.c.d if (_t := a.b)? else None + +While this would work and make it easier to write ``is not None``, +it "only" replaces three short words without meaningfully addressing +the issues described in the `Motivation`_ section. Working with nested +objects with ``optional`` attributes would still require if statements +or expressions. + Builtin function for traversal ------------------------------ @@ -950,6 +975,12 @@ still be treated as values. For example ``math.nan.imag`` is well defined (it is ``0.0``) and so short-circuiting ``math.nan?.imag`` to return ``None`` would be incorrect. +In Python 3.15, :pep:`661` added the option to define custom sentinels +using :py:class:`sentinel`. Some of these could be considered to represent +no value. In general though, custom sentinels are mostly recommended +for cases where ``None`` itself can be a valid value and are therefore +not as common. + As ``None`` is already defined by the language as being the value that represents "no value" the idea is rejected. @@ -1028,7 +1059,7 @@ Change ``primary`` rule to be right-recursive --------------------------------------------- The ``primary`` grammar rule as it is defined [#py_grammar]_ is -left-recursive. This can make it difficult to reason about especially +left-recursive. This can make it difficult to reason about, especially with regards to the `Short-circuiting`_ and `Parenthesized expressions `_ behavior. @@ -1061,7 +1092,7 @@ might prove difficult. Not to mention that the ``NoneAware*`` nodes would contain parts that are both left- and right-recursive which would be confusing. -In comparison the proposed `Grammar changes`_ are intentional kept to +In comparison, the proposed `Grammar changes`_ are intentional kept to a minimum. The ``None``-aware access operators should behave more or less like a drop-in replacement for ``.`` and ``[ ]``, only with the behavior outline in this PEP. @@ -1103,6 +1134,46 @@ split it up the expression, combine both parts with ``and``, add an assignment expression (do not miss the brackets!) and add the ``is not None`` check. The whole process is a lot more involved. +Not pythonic or not explicit enough +----------------------------------- + +Some voiced their concern that the ``None``-aware access operators are +not pythonic or not explicit enough. They prefer the current, in their +opinion, better option of using explicit ``is not None`` checks. + +To evaluate that sentiment, it is helpful to keep in mind that explicit +``is not None`` checks are the best case scenario. As explained `before +`_, developers often choose to use the easier to +write option of repeating variable names and implicit boolean checks. +Additionally, with boolean checks and conditional expressions there is +the challenge of different coding styles. Some prefer ``a and a.b`` +while others choose ``not a or not a.b``, and for conditional expressions +some write ``a.b if a is not None else None`` while others prefer +``None if a is None else a.b``. All that increases complexity and makes +it more difficult to read and comprehend. + +In contrast, the ``None``-aware access operators provide **exactly one** +easy way to write these expressions. When encountering ``?.`` or ``?[ ]``, +readers do not need to check if ``and`` / ``not ... or`` / ``is`` / +``is not`` are chosen correctly or if the implicit boolean check +accidentally matches an unwanted case. It is enough to know that the +left-hand side of the expression at that point can be ``None``, and if +that is the case, the remaining part of the expression will evaluate to +``None`` as well. + +Furthermore, ``?.`` and ``?[ ]`` by themselves do not remove the need to +deal with ``None``. Developers must still choose a way forward. +This could be returning early or raising an exception with an ``is None`` +check, providing an appropriate default value (if :pep:`824` is accepted, +that could be done with ``??``) or passing it along to a function which +accepts ``None`` already. + +Finally, the operators will not change some of the long established +patterns in Python. For example, :py:meth:`dict.get` will still be +necessary to parse dictionaries with absent or missing values. Also, +``?.`` and ``?[ ]`` will not silently catch exceptions, see +`Exception-aware operators`_. + Easy to get ``?.`` wrong ------------------------ @@ -1221,8 +1292,8 @@ could also be used to deal with ``optional`` attributes. However, getting them right can be equally tricky as the conditional expression. There are a few pitfalls to watch out for, only using keyword attributes for class patterns, making sure -the attribute names are correct since the class pattern can also match -``missing`` once and as such will not emit an error if it is misspelled, +the attribute names are correct since class patterns will not emit an +error if they are misspelled and instead just not match the expected case, and the performance impact from an often times unnecessary ``isinstance`` check. Furthermore, great care must be taken during refactorings as patterns often can not be updated automatically. @@ -1230,7 +1301,7 @@ patterns often can not be updated automatically. While the match statement has been available in Python since 3.10, anecdotal evidence suggests that developers still prefer other alternatives for ``optional`` attributes, at least for simple, -straight-forward expression. Pattern matching starts to become much more +straight-forward expressions. Pattern matching starts to become much more useful once multiple attributes or values need to be checked at the same time. @@ -1264,6 +1335,22 @@ get any more IDE suggestions and type checking these expressions is also fairly limited. Furthermore, normal functions do not support short-circuiting for the remaining expression. +The tradeoffs could be acceptable for simple attribute access, like +``"a.b.c"``. However, for short-circuiting to work, the traversal +function would need to accept **all** possible tail variations +including indexing and call expression: + +.. code-block:: python + + traverse( + data, + "a.b[f(x=1, **kwargs)].some_func((y := 2), 3 if y % 2 == 0 else 4)", + ) + +Not only is it a challenge in itself to read the access string without +syntax highlighting, it is also unlikely type checkers will add support +for it. + Proliferation of ``None`` in code bases --------------------------------------- @@ -1296,11 +1383,22 @@ number of other modern programming languages. Furthermore, adding times since :pep:`505` was first proposed over ten years ago. In Python ``None`` is frequently used to indicate the absence of -something better or a missing value, e.g. ``dict.get(key)``. Other +something better or a missing value, e.g. :py:meth:`dict.get`. Other languages do often have two separate sentinels for that, e.g. JavaScript with ``null`` and ``undefined``. This only contributes further to the prevalence of ``None`` in Python. +Use custom sentinels instead of ``None`` +---------------------------------------- + +In Python 3.15, :pep:`661` added the option to define custom sentinels +using :py:class:`sentinel`. This addressed an issue in cases where +``None`` itself is a valid value and thus could not be used as sentinel. + +In general though, ``None`` will still be preferred if a sentinel is +needed, simply because it already exists for that exact purpose and +is easier to use. + ``?`` last available ASCII character ------------------------------------ @@ -1363,6 +1461,17 @@ Footnotes Change History ============== +* 31-Oct-2026: + + - Added `None-aware binary operators`_ and + `Add ? as shortcut for is not None`_ as rejected ideas. + - Mention :pep:`661` Sentinel Values in `No-Value Protocol`_ and added + `Use custom sentinels instead of None`_ to common objections. + - Added `Not pythonic or not explicit enough`_ to common objections. + - Extended Just use `... a traversal library`_ to include a more + complex example which highlights the limitations of traversal + functions. + * 20-Oct-2026: Turned `How to Teach This`_ into a mini-tutorial.