FAQ: Analyze FarmBurg's A/B Test - Performing a Significance Test II

This community-built FAQ covers the “Performing a Significance Test II” exercise from the lesson “Analyze FarmBurg’s A/B Test”.

Paths and Courses
This exercise can be found in the following Codecademy content:

Data Science
Analyze Data with Python

FAQs on the exercise Performing a Significance Test II

There are currently no frequently asked questions associated with this exercise – that’s where you come in! You can contribute to this section by offering your own questions, answers, or clarifications on this exercise. Ask or answer a question by clicking reply (reply) below.

If you’ve had an “aha” moment about the concepts, formatting, syntax, or anything else with this exercise, consider sharing those insights! Teaching others and answering their questions is one of the best ways to learn and stay sharp.

Join the Discussion. Help a fellow learner on their journey.

Ask or answer a question about this exercise by clicking reply (reply) below!
You can also find further discussion and get answers to your questions over in #get-help.

Agree with a comment or answer? Like (like) to up-vote the contribution!

Need broader help or resources? Head to #get-help and #community:tips-and-resources. If you are wanting feedback or inspiration for a project, check out #project.

Looking for motivation to keep learning? Join our wider discussions in #community

Learn more about how to use this guide.

Found a bug? Report it online, or post in #community:Codecademy-Bug-Reporting

Have a question about your account or billing? Reach out to our customer support team!

None of the above? Find out where to ask other questions here!

Hello,

I´ve done the exercise and ther something I don´t understand

We accept the hypotesis null when the price is 4.99

But if you try a random binomial (like: sum(np.random.binomial(1666,83/1583,20000)>=201)/20000) the probability is 0, and it makes sense because you need 201 success when the average success is 83.

How can be posible that the binomial test accept the null hypotesis?

Thanks and regards

Pablo Arias

Hello,

I’ve been stuck in the second test for so long I decided to click on the solutions button. Yet I still don’t understand. Where does the number 316 and 1666 come from?

If x = the number of purchases for Group A, shouldn’t it be 1011 and not 316?
If n = the total number of visitors assigned Group A, shouldn’t it be 4998 and not 1666?
Is there anything I’m missing here?

EDIT: Finally figured it out!

316 came from the ‘Group A is_purchase’ row in the dataset provided at the top of the exercise.
1666 came from either 4998 / 3 OR 316 + 1350.

Thanks :smiley:

Hello,

After the exercise, we know that there is a significant difference between the observed purchase rate and the expected one.
But we CANNOT know that the observed purchase is significantly greater or significantly less than the target percent of purchases.
So, how can we say that the price of $4.99 is the best one we should choose?

Can anyone explain it to me?

Thank you so much.
Best regards./.

1 Like

Hi Igk1910

hope this helps you

We already calculated how many percent we should have, which is approx. 4%

p_clicks_499 = (1000 / 4.99) / num_visits

If we could achieve this conversion rate or above, that means we would be successful
we knew that 83 out of 1666 is 4.98% so want to make sure whether this is fluctuation or significant difference.
In the exercise, we got p-value of 0.045623 so concluded that significantly greater difference

If you simulate this case with using numpy (np.random.binomial(83, 1666, size=10000)), you will be able to see the distribution.
Here’s attached a binomial graph I made.

test
※Blue vertical line indicate x=83, which is number of people who purchased the product

1 Like

Oh, I forgot that there was a ratio of 83 over 1666. Thanks for your help :hugs: !!!