Shell and Unit tests are great options to write flexible, modular tests. Here's how they work:

Define a test case: For Unit tests, you can write a function in the environment native language (currently supports Java/Python), including any required imports. For shell tests, you can write an arbitrarily long shell script.
Return a test result using the codePost TestOutput syntax.

Unit Test Example

Say your students wrote a class called Calculator and you want to test their Calculator.add method. We write a java function to run it on a series of random inputs:

import java.util.Random; 

public class Test {
    public static void testAdd() {     

        Random rand = new Random(); 

        // Iterate over 100 random (x,y) int pairs
        for (int i = 0; i < 100; i++) {
            int x = rand.nextInt(1000);
            int y = rand.nextInt(1000);

            int studentAnswer = Calculator.add(x, y);
            int solution = x + y;
      
            if (studentAnswer !== solution) {
                // report test failure
            }          
        }

        // report test passed
    }
}

We need to figure out how to report whether the test passed or failed to codePost.

TestOutput

In codePost tests, we report outcomes using the TestOutput function. TestOutput is globally available from any code you write, from either the Test Editor or Source Editor. It has the following signature:

TestOutput <passed: boolean> <logs: string OPTIONAL>

Back to the example

Let's fill in our code with calls to TestOutput. In Java Unit tests, we return a TestOutput object.

import java.util.Random; 

public class Test {
    public static void testAdd() {     

        Random rand = new Random(); 

        // Iterate over 100 random ints pairs
        for (int i = 0; i < 100; i++) {
            int x = rand.nextInt(1000);

            int studentAnswer = Calculator.square(x);
            int solution = x * x;
      
            if (studentAnswer !== solution) {
                return new TestOutput(false, "Failed: " + x);
            }          
        }

        return new TestOutput(true);
    }
}

Shell Script example

In shell tests, you can make an equivalent call to TestOutput instead of returning an object.

For example, let's define a shell script test case that makes sure a student's helloWorld.py script outputs the right string:

result=$(python3 helloWorld.py)
if [ "$result" == "Hello World!"]; then TestOutput true "You Passed!!";
else TestOutput false "Wrong output. Received $result"; fi;

In the above code, we run our command python3 helloWorld.py and capture the result.
If the student's result is Hello World , we call TestOutput with a custom log of You Passed!! .
If the student's result is something else, we call TestOutput with a custom log outputting the student's result.

Let's run this on a student's incorrect code:


########### Incorrect helloWorld.py###########
print("Goodbye, World"

A Bash pro-tip: Unlike some languages, spacing is quite important in Bash scripting. For example: when you write if, else statements, make sure there is space between each bracket and the args, i.e., if [ "$result" == "Hello World!" ]; instead of if["$result"=="Hello World!"].

Using TestOutput in File Mode

If you are using File Mode, check our our article on how to use TestOutput syntax in File Mode.