Quick Start

Prerequisites

• Node.js ≥ 20 and npm.
• Amazon Bedrock access — the judge runs on Bedrock (Claude). Have AWS credentials available (e.g. aws configure or environment variables) with Bedrock model access in your region.
• PostgreSQL and Redis — required at runtime. If you use the Docker/Terraform path below, both are started for you.

1. Clone and install

git clone https://github.com/alokkulkarni/aria-evaluator-ts.git
cd aria-evaluator-ts
npm install

From here, choose how you want to run ARIA — plain npm, or Docker via Terraform.

Option A — Full stack with Docker (recommended)

One command brings up the app plus its PostgreSQL and Redis — health-ordered, with the database migrated automatically. Postgres and Redis stay internal to the compose network; only the app is exposed.

# optional: copy config (AWS region, secrets, extra judge keys)
cp .env.example .env

cd infra/terraform/environments/local
terraform init
terraform apply                 # app at http://localhost:3001

Equivalent without Terraform: docker compose up --build from the repo root. Full details are in Deploy with Terraform.

First login: a default admin user is created and its password is printed to the logs — grab it with docker compose logs aria-evaluator | grep -A4 "default admin", then invite teammates from the in-app Team page.

Option B — Run from source with npm

Best for active development. Provide your own host-reachable Postgres + Redis (the compose stack’s are internal-only), then:

cp .env.example .env
# edit .env: DATABASE_URL (Postgres), REDIS_HOST/REDIS_PORT,
# JUDGE_BEDROCK_REGION, and your AWS credentials

npm run db:migrate    # create the schema
npm run dev           # API + dashboard at http://localhost:3001

See Run Locally for the full environment and Configuration for every variable.

Run your first scenario

Open http://localhost:3001, choose a provider and a scenario, and watch the conversation and scores stream in live. Prefer the terminal? Point an adapter CLI at your agent and pass a scenario from scenarios/:

npm run cli:openapi -- --scenarios-dir=./scenarios --scenario=banking/functional/account_query

ARIA runs the conversation, scores the transcript with the judge panel, and writes an HTML + JSON report. See CLI Usage for the other providers.

Next steps

• Write your own scenarios.
• Connect ARIA to your agent with an adapter.
• Understand the 15 dimensions the judge scores.