How Codecademy Checks Your Answers


#1

There are countless ways to write code that gets the job done, that performs a certain task. Thus, there is not always a singular “right answer” to coding questions in general nor for Codecademy questions in particular. How, then, does Codecademy assess what is right and wrong? Codecademy uses multiple tools and tests to check whether your code is “correct” – something we call submission correctness tests or SCTs.

In brief:

  1. We teach you and ask you to do something in a particular way, and then we test if you’ve done what exactly we expected you to do.
    or
  2. We are able to test the result of what you did and see if it “worked” even if you did so in an unusual way.

It isn’t always possible or advisable to follow the second route, particularly when there are multiple ways to solve a problem. Occasionally, even if you happened to get a “right” answer, the way that you got that right answer could lead to some unexpected problems further on in the course. Given that there are so many ways to potentially solve a problem and we can’t write courses based on infinite branching possibilities for code, we use a combination of the two tests. For more info about how this works, read on.

Linting

Linting is the running of a simple program that will check your code for some common, potential, or predictable errors. This may include things like not ending a tag or including a semicolon. A lint program is what will return details for many of your potential missteps with code whether you’re a beginner or an advanced programmer – you’ll find linting tools in many of your favorite text editors like Atom or Sublime Text.

Output Tests

For many tasks, Codecademy is able to assess what your code produces as an output, comparing that with what we’d expect to see with what we asked you to produce. This can be a powerful tool, but it is also a limited one. When there are numerous methods of getting code to do the thing you want to do, it can mean that your code can evolve in a very different way than many other coders’ work, which would give us, over time, an unmanageably large set of potential code solutions to account for when designing our curriculum. If you have multiple steps within a single exercise, we’re probably using this kind of test to assess your answers before you make it to the next exercise.

For example, let’s say that you are on the second checkpoint of a React exercise, below:

For the kinds of tests that happen before the final question in this exercise, there may be no single “answer code” that we are testing your submission against, just the output. This also means that you can often tinker with your code or make large rewrites and still move to the next checkpoint so long as the output that we are checking remains the same. It is also useful to note that, hypothetically, there may be ways to write code that ‘pass the tests’ but aren’t correct per the instructions. Given that you could tweak your code infinitely in this manner, in ways that could affect future exercises, we need another method to keep everything orderly.

Code Tests

The final check Codecademy uses is to verify a single “right answer.” For this, there is an “answer code” – we want to make sure you wrote the code in a very specific way. This test checks the code that you’ve written, not just that code’s output. What we’re looking for here is the code that we would have expected you to write if you followed all of our instructions to the letter. This test does allow for some variations, for example if we ask you to enter your name on line 12, we aren’t expecting that everyone will enter the exact same string . When you press the get code button, we replace your code with the code that we are looking for, thus enabling you to pass the exercise.

Note that this “answer code” will get you past the final checkpoint, but it won’t always allow you to pass all the previous checkpoints one by one. Part of the reason for this is that sometimes those checkpoint instructions ask you to make changes that aren’t reflected in the final code by the time you get to the last checkpoint – for example in checkpoint two you may be asked to revert a change you made in checkpoint one. This is also a major reason why when you press the get code button you are automatically passed through all of the checkpoints in an exercise.