Hi Codecademy team,
In the sample size determination module in DS path, we can calculate the sample size for A/B Tests with the sample size calculator, which requires baseline conversion rate, statistical significance, and minimum detectable effect. Upon reaching, this module about determining lift, I got confused. Is Lift == Minimum Detectable Effect?
I would appreciate a great explanation for this.
Thank you,
Jimmy
Hi @jimmywijaya
It IS confusing and there isn’t much of a discussion about the two (from what I read in the lesson). Plus, it would be super helpful if there was a cheatsheet! (IMO).
No, they’re different.
Lift is your desired conversion rate as calculated from your baseline.
Let’s say your baseline conversion rate for your site is currently 20% and you want to increase that by changing something on your website (or in a marketing campaign, etc.) to 22%.
So, 22%-20% = 2% lift in conversion.
Your MDE is calculated as a percent of the baseline and it’s the amount of effort you need to put forth to arrive at the desired change, or, lift.
MDE = desired conversion rate lift / baseline conversion rate x 100%
So, 2% / 20% = 10% MDE for your experiment.
The MDE is used when designing the A/B test and is the minimum improvement you’d like to see in your hypothesis test.
https://www.invespcro.com/blog/calculating-sample-size-for-an-ab-test/
" The simplest way to think of MDE is the following – it is essentially the smallest possible change in your primary KPI that your experiment can detect with statistical certainty ."
From here: https://www.brooksbell.com/resource/blog/minimum-detectable-effect/
Maybe someone else can chime in if I’m not correct or explaining it well enough?
Hey Lisa,
Thanks for trying to clear things up but I have some doubts from your understanding of what Lift is.
You mentioned that,
Isn’t 2% the conversion rate itself and not the lift?
From reading the Codecademy module again, the lift formula is 100* (new conversion - old conversion) / old conversion
Therefore, from your example, isn’t the lift supposed to be as follow?
100 * (22-20)/20 = 10%
Isn’t this also the same calculation you mentioned to calculate MDE?
Furthermore, when I re-read the sample size determination module page 2, it seems the lift is equal to the MDE?!
Please refer to the small exercise on that page:
The sample size is 760, derived from:
Baseline conversion rate: 20%
Statistical significance: 90%
Minimum detectable effect: 30% (“at least 30% increase”)
See, that’s why I think it’s confusing in the lesson. I even went back over my old notes for it and found what I wrote to be incomplete and not what I’ve read on other sites.
They are two different things. If you google “lift” and “MDE”, they’re two different things.
It’s my understanding that lift is your desired outcome–the change that you want to see. 20%–>22%. that’ 2% change is lift.
#7, specifically- MDE:
https://www.optimizesmart.com/understanding-ab-testing-statistics-to-get-real-lift-in-conversions/
Thanks Lisa for this. I am gonna put a pause on getting to the bottom of it.
Wish the Codecademy DS Instructor can help us with this. Plus, they definitely need to create a cheat sheet for this particular module to help our learning journey 
You’re welcome! We’re here to learn. Did you complete any of the projects on the DS path or DA path? There’s the National Parks one, Muscle Hub A/B test specifically.
A cheatsheet would be incredibly helpful. I’m going to look into that.
It seems the more that I investigate these terms/concepts, the more grey areas I uncover. Haha. 
Hey Lisa,
Thanks for that, I will look forward to the cheat sheet and yes I have done the National Parks one, but not the Muscle Hub yet.
I completely agree, the more I google the difference of those two concepts, the more confusion I get. I think its better to press pause on these two concepts. I need to move on my learning journey.
I also just realised that I have spend 2,5 weeks on Hypothesis testing and Sample Size Determination. I think I spend too much time on these two haha.
Regards,
Jimmy