Dictionaries with sympy symbols as keys throw TypeError

[notPlancha]

from icecream import ic
from sympy import *

x, y = symbols("x y")

d = {x: "hello", y: "world"}
ic(d)

results in TypeError: cannot determine truth value of Relational due to pprint's pformat (according to the stack), since it tries to sort the keys. This doesn't happen with normal objects as keys for some reason.

This, for example, makes it so you can call ic for system of equations:

from icecream import ic
from sympy import *
from sympy.abc import x, y

res = solve([x + 2, y - 2])
ic(res) # TypeError

[notPlancha]

related #103, meaning that ic.configureOutput(argToStringFunction=lambda x: pprint.pformat(x, sort_dicts=False)) is a temporary fix

[gruns]

@notPlancha thank you for the issue and posting the workaround 🙏

sorting dictionaries is useful for human debuggers. ie us. 🙂

ive never used sympy before. any idea why the keys cant be sorted? is this an issue better filed with them to support sorting?

[notPlancha]

The keys can't be sorted because there's no order to the symbols, like we can't say if x > y. It's not really an error on their part, it's on purpose.

I feel like if this is something that icecream could address, perhaps a check if the keys can be sorted or not (with for example a try except with the sorted before calling pprint could work). Without it, users can find this and assume the bug is either because of something they did or the original library, since it's rare for people to debug the debugging tool while trying to debug their code.

[salabim]

I recommend looking at the peek package which does it all ... and more.
See www.salabim.org/peek

[gruns]

I feel like if this is something that icecream could address, perhaps a check if the keys can be sorted or not (with for example a try except with the sorted before calling pprint could work).

yep! first thing that comes to mind is:

• icecream should attempt to sort the dict
• if the dict can be sorted? great! sort the dict and output the sorted dict
• if the dict can't be sorted:
  • print a warning that the dictionary cant be sorted
  • output the dictionary unsorted

but breaking on a failed dict sort is not desired behavior

@notPlancha do you have time to take a stab at implementing this and submitting a PR? 🙌 shouldn't be too bad beyond a try + except wrapper. probably 4-10 line diff total

[notPlancha]

I was trying that but upon further inspection it seems that pprint already has rail gaits for these types of situations, which is why it is able to sort objects or complex numbers as keys.

on pprint.py#L84:

class _safe_key:
    """Helper function for key functions when sorting unorderable objects.

    The wrapped-object will fallback to a Py2.x style comparison for
    unorderable types (sorting first comparing the type name and then by
    the obj ids).  Does not work recursively, so dict.items() must have
    _safe_key applied to both the key and the value.

    """

    __slots__ = ['obj']

    def __init__(self, obj):
        self.obj = obj

    def __lt__(self, other):
        try:
            return self.obj < other.obj
        except TypeError:
            return ((str(type(self.obj)), id(self.obj)) < \
                    (str(type(other.obj)), id(other.obj)))

Applying this would be a better solution since it still sorts regardless; what I am wondering is why these guardrails fail for sympy.

Aditionally, sympy's own pretty_print does not raise the same issue as pprint does, and it does sort dictionaries by default as well:

from sympy import symbols, pprint as sym_pprint
from pprint import pprint
x, y = symbols('x y')
d = {y: 1, x: 2}
sym_pprint(d) # {x: y, y: 1}
pprint(d)          # TypeError

This leads me to believe that this is either an issue on sympy's behalf, or it's due to another reason besides sorting. I'll investigate this further tomorrow

[salabim]

Why not simply use pprint/pformat with sort_dicts=False?

[notPlancha]

Got it: pprint's _safe_key lt returns sympy's x < sympy's y because it's a valid operation and it actually returns; unexpectibely for pprint, it's an issue since the operation (purposely) doesn't return a boolean, but instead a Relational, that sorted will try to call bool on it. The issue is that because it returns safely without raising any exception, it only raises an issue after it leaves the try ... except block.

from sympy.abc import x, y
x < y # x < y
type(x < y) # <class 'sympy.core.relational.StrictLessThan'>
bool(x < y) # TypeError

Checking out the docs for operator.py, it seems it's intentional that the lt operator can return non boolean values. To fix this, IMO best option would be to change pprint.py:

class _safe_key:
    __slots__ = ['obj']

    def __init__(self, obj):
        self.obj = obj

    def __lt__(self, other):
        try:
-            return self.obj < other.obj
+            return bool(self.obj < other.obj)
        except TypeError:
            return ((str(type(self.obj)), id(self.obj)) < \
                    (str(type(other.obj)), id(other.obj)))

I think this is unfeasible since (not only is it scary to contribute to cpython directly, but also that) changing pprint now would not change supported icecream python versions. Worth a shot to anyone brave enough.

Second best option is to do a try except for icecream, like originally planned, but right before the actual pformat call, and turn off sorting after a TypeError. If that doesn't resolve it then it'll still raise an Exception, just twice.