Semantics of operation calls without arguments

[ebonnal]

@mjkanji Following up on our discussion in #57 and #58.

We have operations for which the behavior is hopefully intuitive when passing zero arguments:

• flatten()
• .observe()
• .group() -> groups all the elements together into a single list
• .distinct() -> deduplication on elements themselves, like .distinct(key=lambda _: _).
• .filter() -> filter truthy elems, like .filter(None) or .filter(lambda _: _)

We have operations for which there is no intuitive behavior when passing zero arguments and the first positional argument should be mandatory:

• .foreach() (applies what?)
• .aforeach() (applies what?)
• .map() (maps what?)
• .amap() (maps what?)
• .groupby() (groups by what?)
• .catch() (catches nothing? catches everything?)

no-op

.throttle() and also .truncate()/.skip() after #59, are valid and no-op, but while intentional .truncate(None) and .skip(None) make sense to be no-op, .truncate()/.skip()/.throttle() are more likely developer mistakes that need to be notified via an error, right?

[mjkanji]

I'm never developed a user-facing package so I'm not quite sure. It's definitely an interesting UX/DX issue.

What I know for sure:

• .catch() should raise an error. Catching all exceptions (as is the case now), should always be a conscious decision. This will be a breaking change, I guess.

Some random thoughts (these could all be individual issues):

.catch()

1. What's the correct way to catch nothing? In my testing, it seems .catch(None) does this — it'd be good to document this in the docstring.

2. kind and when are currently ANDed together. It might be interesting to have an alternate API where .catch(when: Exception | Callable)

where if given a function, it directly evaluates the function on the Exception.

This'd also enable catching multiple exceptions with the following:

stream.catch(lambda x: isinstance(x, (TypeError, ZeroDivisionError))

And we could potentially also unify the suggestion from #58 such that:

def catch(when: Exception | Iterable[Exception] | Callable):
  ... 

The current kind + when behaviour can be implemented by just stuffing multiple conditions in the function:

.catch(lambda x: isinstance(x, ConnectionError) and "max retries" in str(x))

And catch everything/nothing:

.catch(lambda x: True) # catch everything
.catch(lambda x: False) # catch nothing

It'd also allow for creating side-effects when catching exceptions (e.g., logging the exception to a remote server before replacing it) — a la #62.

.filter()

You said that you find it intuitive that .filter() -> filter truthy elems. I don't. I'd think that .filter() should filter nothing and should be equivalent to .filter(lambda _: True).

So, maybe this also need to have a required first argument.

For the rest, it makes sense to not allow zero arguments, as you've said.

[ebonnal]

Thank you for your detailed message @mjkanji 🙏🏻

.catch() should raise an error. Catching all exceptions (as is the case now), should always be a conscious decision. This will be a breaking change, I guess.

Agree, it’s a breaking change, so it will need to be bundled in the next minor version update (1.5.0). The good thing is that it doesn’t modify silently the behavior of the affected existing code; it simply makes it break.

What's the correct way to catch nothing? In my testing, it seems .catch(None) does this — it'd be good to document this in the docstring.

Current typing forbids to pass a None to .catch so it is not on-purpose that .catch(None) behaves that way, nice finding, and I actually find this really intuitive. Let's make the future mandatory first arg an Optional[Exception] 👌🏻

def catch(when: Exception | Iterable[Exception] | Callable)

Interesting, I will take more time to think about this .catch revamp, will ping you on the future PR to iterate!

You said that you find it intuitive that .filter() -> filter truthy elems. I don't. I'd think that .filter() should filter nothing and should be equivalent to .filter(lambda _: True).

but in at the same time .filter() is super close to .filter(None) which iself should behave the same as the builtin filter(None, iterable) i.e. filtering truthy elems. Curious to have your take on that, given the similarity with builtin filter 🙏🏻