Environment: 

  • Language: Python 3.7

  • Build Type: Default

  • Packages: None

  • Run Script: None

  • Helper Files: Upload your Test script as a helper file

Running tests

There are two ways to run an existing unittest script in codePost:

Option 1: Dumping the script output to a _tests.txt file
Option 2: Storing the results as codePost test objects

Option 1: Dumping the script output to a _tests.txt file

Tests

  1. Open file mode, and click "Add File". Create a new Test File

  2. In your Test File, add the command: python3 <scriptName>.py 

Settings

  • Turn on "dump test outputs to  _tests.txt. This redirects the unittest outputs into a file in each student's submissions

  • If you want students to see the output of all tests when they submit, you'll also want to turn on "Expose outputs to students on submit".

Result

  • Whenever you run tests on a submission, it will have a file _tests.txt that have the outputs of the test script 

Option 2: Storing the results as codePost test objects

There are a lot of benefits of storing test results as codePost objects, including attaching metadata (points, explanations) to each test, as well as exposing a subset of tests on student submit.

To do this, we override the unittest success and failure reporting functions in order to report the results to codePost on pass and fail.

Tests

  1. Open file mode, and click "Add File". Create a new Test File

  2. In your Test File, add the command: python3 <scriptName>.py 

  3. Edit your test script to add the following snippet to the top of the script

from subprocess import Popen

## Function to report test results: ###
## Learn more: https://app.intercom.io/a/apps/kg4u5rp1/articles/articles/3715700/show?language=en ###
def report_result(category, case, res, logs=''):
  reporter = ("TestOutput \"%s\" \"%s\" %s \"%s\"") % (category, case, "true" if res else "false", logs)
  Popen(['/bin/bash', '-c', reporter])

## Unittest override functions ###
class MyTestResult(unittest.TestResult):
    def addFailure(self, test, err):
        # Report to codePost the test failure
        report_result("<CategoryName>", str(test), False, str(err))
        super(MyTestResult, self).addFailure(test, err)

    def addError(self, test, err):
        # Report to codePost the test error
        report_result("<CategoryName>", str(test), False, str(err))
        super(MyTestResult, self).addError(test, err)
       
    def addSuccess(self, test):
         # Report to codePost the test success
        report_result("<CategoryName>", str(test), True, "")
        super(MyTestResult, self).addSuccess(test)

Notice that each of the report_result calls in the override functions have an argument "<Category Name>". This is the category that the tests will be nested under, so you might want to change the name. 

Settings

  • Turn off "Parse TestOutput calls in source editor". This is important, as it tells codePost not to delete the tests you've programmatically created if it doesn't find it explicitly in your code!

Result

  • Whenever you run tests on a submission, it will create a test result for that submission that matches the result of the unit test. Any errors will be in the logs of that test. 

  • You can also selectively expose tests to be run when students submit in the Test Editor!

Did this answer your question?