I/O tests are the simplest tests in codePost. They require no code to write or maintain! All you need to do is fill out a few fields, and we take care of everything else.
I/O tests consist of three components:
Generic test fields (name, points, explanation)
A command to run: Specified either through File Syntax or Command Line Syntax. File Syntax is useful for testing a specific function, and Command Line Syntax is useful to running main functions, or passing in data from the command line.
An expected result
When you run an I/O test, it runs the command in the same container as student code and checks to see if the result of the command matches the expected result.
If the result matches the expected result, the student's code Passes the test.
If the result doesn't match the expected result, the student code Fails the test.
Our student has submitted a python file Calculator.py that has two functions to add and subtract , and a main function that takes in a set of operations and outputs a result. Let's go ahead and make a few I/O tests to test this code.
We want to make the following I/O tests:
A test to check
addA test to run the
mainmethod with a helper filedata.txt
File Syntax
To test our file add , let's us the File Syntax for an I/O command. When we select "File" in an I/O test, we see the following fields:
(1) File: Whether File Syntax or Command Line Syntax. Because we want File Syntax, we've selected File
(2) File name: the name of the file we want to run a function for. You can select a name from a list of uploaded solution code files. Here we've selected Calculator.py
(3) Function name: the name of the function we want to run. For some languages, this will be pre-populated based on functions parsed from your file. We've inputted add .
(4) Arguments: the arguments to run your function with. Here we've chosen 1,2 . You can also input more complex objects in this field. For example, if we wanted to pass a 2D int array in java, we would input the following; new int[][] {{1,2},{3,4}}
(5) Return / Output: whether we want to test the value a function returns, or the value a function outputs to stdout. If we wanted to check whether our function prints out a value, we would select output; if we want to check whether our function returns a value, we would select return. In this case, we choose return .
(6) Expected result: the result we expect. In this case, we expect add(1,2) to return 3 .
When we run this, we see that the function passed!
Let's see what it looks like it fails. We change add to multiply and see the following result:
Command Line Syntax
File Syntax is great for testing individual functions, but there are some scenarios where we want to be able to use the command line. Shell Scripts and Unit Tests are great alternatives for more complex tests, but we can use Command Line syntax to use the command line while keeping our tests simple. Some examples might be:
Piping a data file to a student function
Running a main function
Stringing multiple commands together (compile, then run)
Let's run our Calculator.py main function with a datafile data.txt to see how this syntax works.
First we upload data.txt as a custom dependency:
Then we write our test:
(1) Command Line: Whether File Syntax or Command Line Syntax. Because we want Command Line Syntax, we've selected Command Line
(2) Command: the text to run in the command line. Here we run python3 Calculator.py < data.txt to run the student's main file with our piped in data
(3) Expected result: the result we expect. In this case, we expect the code to return ((10 + 4 - 2) * 7) / 3 = 28.0 .