FAQ: Analyze FarmBurg's A/B Test - Performing a Significance Test

This community-built FAQ covers the “Performing a Significance Test” exercise from the lesson “Analyze FarmBurg’s A/B Test”.

Paths and Courses
This exercise can be found in the following Codecademy content:

Data Science
Analyze Data with Python

FAQs on the exercise Performing a Significance Test

There are currently no frequently asked questions associated with this exercise – that’s where you come in! You can contribute to this section by offering your own questions, answers, or clarifications on this exercise. Ask or answer a question by clicking reply (reply) below.

If you’ve had an “aha” moment about the concepts, formatting, syntax, or anything else with this exercise, consider sharing those insights! Teaching others and answering their questions is one of the best ways to learn and stay sharp.

Join the Discussion. Help a fellow learner on their journey.

Ask or answer a question about this exercise by clicking reply (reply) below!
You can also find further discussion and get answers to your questions over in #get-help.

Agree with a comment or answer? Like (like) to up-vote the contribution!

Need broader help or resources? Head to #get-help and #community:tips-and-resources. If you are wanting feedback or inspiration for a project, check out #project.

Looking for motivation to keep learning? Join our wider discussions in #community

Learn more about how to use this guide.

Found a bug? Report it online, or post in #community:Codecademy-Bug-Reporting

Have a question about your account or billing? Reach out to our customer support team!

None of the above? Find out where to ask other questions here!

This module tells you to fill in all the values on the contingency table but all of your code from the previous module is gone. Thus you need to back space to find the numbers that you need to enter.

1 Like

They have given you the data from your previous exercise in the table in the ‘Learn’ section. Just look up that table and fill in the contingency table :))

I don’t understand why they tell us to do chi-test.
And I don’t think this test was explained earlier.

I read a bit about it and it seems that in A/B testing most people uses ether straightforward binomial test or t-test if values are not binomial.
It’s all rather confusing, which test and why should be used.

I also had to backtrack to copy my code from the previous module step. I think this particular section needs to be revised by the curriculum editors.

In a contingency table, how does it affect the test results if rows are test groups and columns are outcomes vs. if rows are outcomes and columns are test groups (if at all)? Thanks

1 Like

Transposing the contingency table does not essentially change the result, since only the independence of row and column variables matters in the chi-square test.

Regarding question 4 in this excercise. I couldn’t think of a boolean way to get a variable be either yes or no, so looked up the suggested solution.
The suggested solution is this: is_significant=True
How does this work? This does not, to my eyes, seem to define that it is supposed to be looking at p value or that what determines it being True or False (95% significance level in this case). Please shed some light on this.