2024/02/06/sklearn-pipeline

[giscus[bot]]

2024/02/06/sklearn-pipeline

https://it176131.github.io/2024/02/06/sklearn-pipeline.html

[BenKite]

I have always wondered why we install scikit-learn… but import sklearn. I wonder if that is worth mentioning for those new to Python.

[BenKite]

You mentioned mean imputation towards the beginning. Does sklearn have any multiple imputation approaches that you are aware of? So instead of filling in a single value we will it in with some noise to add uncertainty around what we didn’t observe?

[it176131]

Thanks for the questions, @BenKite!

I have always wondered why we install scikit-learn… but import sklearn. I wonder if that is worth mentioning for those new to Python.

scikit-learn is the (current) name of the library/package/distribution, and sklearn is the name of the directory in the source code. They both refer to the same thing: "SciPy Toolkit for Machine Learning". We pip install scikit-learn because it is the name given to the distribution on the PyPI site, and import sklearn because it is the name of the directory. In the past you could pip install sklearn and it would install scikit-learn for you, but this was deprecated. On the other side, if you try to import scikit-learn, you will raise a SyntaxError due to the hyphen. Technical details 🤓.

You mentioned mean imputation towards the beginning. Does sklearn have any multiple imputation approaches that you are aware of? So instead of filling in a single value we will it in with some noise to add uncertainty around what we didn’t observe?

As far as I know, scikit-learn has two stable imputation classes and one that's experimental. I use both stable classes in the post (see Multi-step Transformation for an example of the SimpleImputer and Custom Transformer for an example of the KNNImputer) but both are for single imputation. The experimental IterativeImputer is more flexible than the other two and could be used for multiple imputation with a little grunt work.

[Mrwent21]

Might actually try to build something with this