Unit Testing with Jest and CI

Once you start to use deliver applications that include AI models for key functionality, regression becomes a significant concern.

There are many unit test frameworks available. Okareo works with any framework abd can even operate standalone via the CLI. Here we demonstrate how you can use Jest to drive evaluation of model behavior during development or in CI.

tip

This page assumes you have a working application and have read and setup Okareo for Typescript. See the Typescipt SDK for more information.

Overview

The Okareo unit tests can appear in a variety of repo locations. We find that often models are part of a larger applicaiton context. When this is the case, it is beneficial to run model evaluations and scenario expansion as part of your general CI/CD process.

In all cases we suggest isolating the model unit tests from other types of tests such as logical, component or functional. Model tests can be time consuming and thought should be given to organizing model tests into those that help in development, as CI checkpoints and as gatekeepers for CD.

This guide provies a simple isolation of model tests into a unique configuration passed to Jest. With Jest, you can add a dedicated jest.model-config.js file and a unique script entry in package.json. The specific name of your config is dependent on your Jest setup.

We like to keep our scenario generation and model finetuning/testing scripts separate. Here we've made a unique location for them at ci-models at the root of the project.

tip

For a running cookbook example of Okareo Typescript + Jest, you can refer to the okareo-cookbook repo.

Add a unique jest entry point in your package.json that you will use to drive the Okareo unit evaluations

package.json

"scripts": {
    "build": "next build",
    "start": "next start",
    "jest": "jest",
    "jest:model": "jest ci-models --config ./jest.model-config.js",
},

Create a configuration file that provides details about how to run your Okareo tests. In this case, the timeout is a critical difference from component or logical unit tests. The file jest.model-config.ts should be in the same directory as your package.json configuration.

jest.model-config.ts

// File: jest.model-config.ts
// Add this file in the same location as package.json
module.exports = {
    preset: 'ts-jest',
    testEnvironment: 'node',
    testTimeout: 600000, // 1 minute (model evaluation can be lengthy)
};

The specific command is based on yarn vs npm and how you named your test.

yarn
npm

yarn run jest:model

npm run jest:model

More coming to this page. Don't hesitate to ask for additional information about other unit testing frameworks or methodologies

Unit Testing with Jest and CI

Overview​

Overview