Implementing Cohen's Dz for paired sample T-Test

[PibeChorro]

Hallo Raphael,
first of - thanks for providing this great library.

I was wondering if you could implement the Cohen's Dz for paired samples. There are already some (closed) issues that your calculation of Cohen's D differs from other packages or from R and that you have good reasons to use the formula you are using.
But since Cohen's Dz is simply calculated by

T/sqrt(N)
or
X_diff.mean() / X_diff.std()

and is also the one often reported (at least in my field) and also the one you find when looking for effect sized on Wikipedia, I would really appreciate it.

Best
Vincent

[raphaelvallat]

Hi Vincent,

Thanks for opening the issue. While it is straightforward to add the Cohen Dz to the low-level compute_effsize function, there's some work in making that new metric available in higher-level functions such as pairwise_tests.

I think we would need to expose a new keyword-argument to all impacted functions, e.g.

pairwise_test(..., paired_cohen_type="avg")
pairwise_test(..., paired_cohen_type="dz")

Alternatively, we could define this as a global option.

Curious to hear what you and other thinks here.

Best
Raphael

[PibeChorro]

Hey Raphael,

... there's some work in making that new metric available in higher-level functions ...

yeah that makes sense. I don't have a good overview of your project nor am I a good coder, so I don't wanna make assumptions about the best solution. However, it seems counterintuitive to me to have a paired_cohen_type argument, when a user chooses a different effect size, e.g., Hedge's g ...

Generally, to me it seems that Cohen's Dz is better known than Cohen's D_avg and I would go for dz as the standard.

In the end it's your choice.

Best, Vincent

[remrama]

Would it be okay to just add something like cohen_z to the low-level compute_effsize? Then users could pass that into the effsize argument in pairwise_test and related functions, so both Cohen's are available (separately) and other effect sizes as well. Maybe if there is concern about users requesting inappropriate effect sizes given a certain stat, a warning could be raised.

[PibeChorro]

Yes I think that would be a great solution

[raphaelvallat]

Agreed, @remrama!