Automated Unit testing with codity

Chaitanya Tyagi·November 20, 2025

Overview

Codity automatically generates comprehensive unit tests for your code changes and can create pull requests with those tests. To maximize the benefits of automated testing, it's recommended to set up continuous integration (CI) to run these tests automatically.

How It Works

Test Generation: When Codity analyzes your PR, it detects changes in supported languages and writes relevant tests for the PR (Go, Python, Ruby)
PR Creation: Generated tests are committed to a new branch and a PR is created
CI Execution: If you have CI configured, tests run automatically on the PR
Auto-Fix: Codity monitors CI status and automatically fixes failing tests (up to 3 attempts)
Merge: Once all tests pass, you can review and merge the test PR

Supported Languages

Go - Generates *_test.go files using testing package and testify assertions
Python - Generates test_*.py files using pytest framework
Ruby - Generates *_spec.rb files using RSpec framework

Setting Up CI for Automated Testing

Required GitHub App Permissions

Before setting up CI workflows, ensure the Codity GitHub App has the necessary permissions to create PRs and commit tests:

Go to your repository settings → GitHub Apps → Codity
Ensure the following permissions are enabled:
- Read access to metadata (required)
- Read and write access to checks, contents, issues, pull requests, and workflows (required for test generation and auto-fix)

Without these permissions, Codity won't be able to create test PRs or push fixes.

CI Workflow Configuration

Below are minimal GitHub Actions workflow files that will run unit tests for each supported language. These workflows trigger on:

New pull requests (opened)
New commits to existing PRs (synchronize)
Reopened pull requests (reopened)

Go Testing Workflow

Create .github/workflows/go-tests.yml:

# Minimal Go Tests CI
# Works for any repository structure with go.mod at root or in subdirectories

name: Go Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Go
      uses: actions/setup-go@v5
      with:
        go-version: '1.21'

    - name: Run tests
      run: |
        # Find all directories with go.mod and run tests
        # Fail the entire job if any test fails
        failed=0
        while IFS= read -r -d '' gomod; do
          dir=$(dirname "$gomod")
          echo "Testing in $dir"
          (cd "$dir" && go mod download && go test -v ./...) || failed=1
        done < <(find . -name go.mod -print0)
        exit $failed

Python Testing Workflow

Create .github/workflows/python-tests.yml:

# Minimal Python Tests CI
# Works for any repository structure with Python code

name: Python Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.11'

    - name: Install dependencies and run tests
      run: |
        python -m pip install --upgrade pip
        pip install pytest pytest-cov

        # Install all requirements.txt files found
        while IFS= read -r -d '' reqfile; do
          echo "Installing from $reqfile"
          pip install -r "$reqfile"
        done < <(find . -name requirements.txt -print0)

        # Run pytest - it will fail if no tests found or tests fail
        pytest -v

Ruby Testing Workflow

Create .github/workflows/ruby-tests.yml:

# Minimal Ruby Tests CI
# Works for any repository structure with Gemfile

name: Ruby Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v4

    - name: Set up Ruby
      uses: ruby/setup-ruby@v1
      with:
        ruby-version: '3.2'

    - name: Run tests
      run: |
        # Find all directories with Gemfile and run tests
        # Fail the entire job if any test fails
        failed=0
        while IFS= read -r -d '' gemfile; do
          dir=$(dirname "$gemfile")
          echo "Testing in $dir"
          (cd "$dir" && bundle install && bundle exec rspec) || failed=1
        done < <(find . -name Gemfile -print0)
        exit $failed

Multi-Language Projects

If your project uses multiple languages, you can combine workflows or create separate workflow files for each language. GitHub Actions will run all applicable workflows based on the files changed.

Example combined workflow:

name: Multi-Language Tests

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  go-tests:
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, '.go')
    steps:
      # Go test steps here

  python-tests:
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, '.py')
    steps:
      # Python test steps here

  ruby-tests:
    runs-on: ubuntu-latest
    if: contains(github.event.pull_request.changed_files, '.rb')
    steps:
      # Ruby test steps here

Benefits of CI Integration

Immediate Feedback: Know instantly if generated tests pass
Auto-Fix: Codity automatically fixes failing tests when CI is configured
Quality Assurance: Ensures all tests are verified before merge
Team Collaboration: Everyone can see test results in the PR

Customization

You can customize these workflows to:

Add code quality checks (linting, formatting)
Upload coverage reports to services like Codecov
Run additional security scans
Deploy to staging environments
Add required checks before merge

Troubleshooting

Tests not running?

Verify workflow files are in .github/workflows/ directory
Check that workflow triggers match PR events
Review Actions tab in GitHub for error messages

Auto-fix not working?

Ensure CI workflows are properly configured
Check that workflow names contain keywords: 'test', 'ci', or 'build'
Verify Codity has access to read CI status