AI & background jobs

Use the shared AI adapter for model calls and the durable job system for slow work. They solve different problems: a queue does not itself control model spending.

Model calls

text = await kit.ai.complete("Summarize this note", effort="low")
record = await kit.ai.extract("Name: Ada", ContactModel)
label = await kit.ai.classify("A sample message", ["work", "personal"])

The service also supports tool-using agents and streaming. Use the platform model catalog (at.ai.models()) and request-setting helpers; do not hardcode per-model reasoning parameters in each app.

Calls record app attribution, usage, latency, and cost estimates. Public apps declaring the ai capability must configure a positive app or default daily AI budget before preflight accepts them. Authorization and budget admission still belong in the execution path.

Keep model runs explicit. A documentation example should not start inference or training simply because the page is opened.

Durable background work

Define a handler in the app's jobs.py:

from at.jobs import job

@job
async def summarize_batch(ctx, batch_id: str):
    # Read inputs using the verified owner passed through the job context.
    owner = ctx["owner"]
    await ctx["set_progress"]({"stage": "started"})
    # Perform idempotent work; store large outputs in S3.
    return {"batch_id": batch_id, "owner": owner}

Enqueue and inspect it from an authenticated route:

user = await kit.auth.require(request)
jobs = kit.jobs.for_owner(user.id)
job_id = await jobs.enqueue("summarize_batch", batch_id="batch-123")
status = await jobs.status(job_id)
await jobs.cancel(job_id)

The production backend uses a PostgreSQL ledger and transactional outbox, then SQS delivery to workers. Job results and progress are scoped to the app and owner. Large inputs and results should be stored in S3 and passed by reference.

Retries, cancellation, and limits

SQS delivery is at least once. A job can be attempted again after a crash or timeout. Database leases fence stale worker results, but cannot undo an email, provider call, or other external effect already performed. Handlers must make those effects idempotent.

Default admission limits are 20 outstanding jobs per owner/app and 100 per app. Four shared worker slots run concurrently. Queue order is not a guarantee of completion order.

Cancellation fences result writes immediately. Running work is stopped when its heartbeat notices lost ownership, normally within about 20 seconds. It is not a rollback of prior external effects.

Arena workloads

Arena has its own queue and worker for completed-match exports and scripted-agent evaluations. Its existing learning workflows remain separate. Do not assume every long-running task in an app has already moved to SQS.

Operator procedures for failures and recovery live in the runbooks.