Skip to content

Compliance testing

Run compliance tests to verify your server correctly implements the GA4GH sequence collections API.

Prerequisites

  1. Install pytest (if not already installed): 

    pip install pytest

  2. Clone the refget repository (tests are in the repo, not the installed package): 

    git clone https://github.com/refgenie/refget.git
cd refget

  3. Load test data into your server from the test_fasta folder

Running the tests

Basic usage:

# Test the default public server
pytest tests/api

# Test a custom server
pytest tests/api --api-root http://127.0.0.1:8100

# Skip sorted_name_length_pairs tests
pytest tests/api --api-root https://your-server.com --no-snlp

Complete example

Test your own server implementation:

# 1. Load test FASTA files into your database
refget admin load test_fasta/base.fa
refget admin load test_fasta/different_names.fa
# ... load other test files

# 2. Start your server
uvicorn myapp:app --port 8100

# 3. Run compliance tests
pytest tests/api --api-root http://127.0.0.1:8100

Test the public server:

pytest tests/api --api-root https://seqcolapi.databio.org --no-snlp

Troubleshooting

"Collection not found" errors

  • Ensure you've loaded all test FASTA files from test_fasta/ into your database
  • Check that your server is running and accessible at the specified URL
  • Verify your database connection settings

Comparison tests failing

  • The comparison endpoint must return the exact format specified in the GA4GH spec
  • Check that arrays, elements, and a_and_b_same_order fields are present
  • Ensure attribute digests match between collections

sorted_name_length_pairs failures

  • This is an optional but recommended attribute
  • Use --no-snlp to skip these tests if your server doesn't implement it
  • If implementing, ensure the digest is computed correctly (sorted array of name-length pair digests)

Connection errors

  • Verify the server URL is correct (include protocol: http:// or https://)
  • Check firewall settings if testing a remote server
  • Ensure the server is running and responding to requests

Test output

Successful tests show:

tests/api/test_compliance.py::test_collection_retrieval PASSED
tests/api/test_compliance.py::test_comparison PASSED
...

Failed tests include details about what went wrong:

FAILED tests/api/test_compliance.py::test_comparison - AssertionError: ...

Use -v for verbose output or -vv for extra details.

What the tests verify

The compliance tests check:

  • Collection retrieval: Can retrieve collections at level 1 and level 2
  • Attribute retrieval: Can retrieve individual attributes by digest
  • Comparison endpoint: Comparison function returns correct results
  • Sorted name-length pairs: Optional attribute for coordinate system compatibility