David B Rosen (PhD) – Medium

David B Rosen (PhD)

Pinned

Published in
TDS Archive

How To Deal With Imbalanced Classification, Without Re-balancing the Data

Before considering over-sampling your data, try simply tuning your classification decision threshold

Aug 2, 2020

Balance

Aug 2, 2020

Pinned

Published in
TDS Archive

How to calculate confidence intervals for performance metrics using an automatic bootstrap method

Are your model performance measurements very precise due to a “large” test set, or very uncertain due to a “small” or imbalanced test…

Sep 8, 2021

How to calculate confidence intervals for performance metrics using an automatic bootstrap method

Sep 8, 2021

Pinned

Published in
TDS Archive

How to Auto-Detect the Date/Datetime Columns & Set Their Datatype When Reading a CSV File in Pandas

When read_csv( ) reads “2021-03-04” and “2021-03-04 21:37:01.123” as mere “object” types, often you can auto-convert them all to datetimes.

Aug 16, 2021

How to Auto-Detect the Date/Datetime Columns & Set Their Datatype When Reading a CSV File in Pandas

Aug 16, 2021

Three more tips:

1. Find and copy Unicode math symbols on Mathematical operators and symbols in Unicode - Wikipedia

Dec 16, 2023

Dec 16, 2023

R² is completely equivalent to MSE in that it is simply a (negatively) scaled and offset form of…

Aug 29, 2022

Aug 29, 2022

Good article!

How To Deal With Imbalanced Classification, Without Re-balancing the Data (Before considering over-sampling your data, try simply tuning…

Aug 29, 2022

Aug 29, 2022

Good article!

As an alternative to using a specific set of functional forms for the relationships, we could use the ROC area of each feature as a "score"…

Jun 7, 2022

Jun 7, 2022

Good article -- I was not aware of how flawed the mean decrease in feature impurity method could be!

One pitfall of permutation feature importance could be if there are some important subsets of the features that are highly-correlated or…

Jun 7, 2022

Jun 7, 2022

The cost to the business of false positives vs false negatives is unlikely to be equal, so you need…

Also it is incorrect to apply ordinary cross-validation to an already-oversampled training set (even oversampled by adasyn or smote)…

Jun 3, 2022

Jun 3, 2022

You mention the average squared error -- how is this defined if it is different from mean squared…

Also it is worth mentioning that R-squared (coeff. of determination) is a rescaled version of MSE such that 100% is perfection and 0%…

Nov 1, 2021

Nov 1, 2021

David B Rosen (PhD)

David B Rosen (PhD)

Lead Data Scientist for Automated Credit Approval (WW) at (not speaking for) IBM. linkedin.com/in/rosen1/

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams