OPA Decision Cycle and Data Loading
Open Policy Agent evaluates authorization by combining three inputs: the policy written in Rego, the data provided as JSON documents, and the query input typically sent as an HTTP POST body. The engine first compiles all Rego files into an intermediate representation, indexes the data for fast lookups, then resolves rules against the provided input. Evaluation is deterministic and side-effect free.
The standard production deployment uses the OPA Docker image openpolicyagent/opa:0.68.0 started with opa run --server --addr :8181 --log-level info. This starts an HTTP server on port 8181. Data can be loaded at startup with --bundle /policies or dynamically via POST /v1/data endpoints. For periodic refreshes, configure a bundle from an HTTPS endpoint that serves a .tar.gz containing both .rego policy files and data.json files.
A production config.yaml for OPA includes:
services:
- name: bundle-service
url: https://bundles.example.com
credentials:
bearer:
token: "secret-token"
bundles:
authz:
service: bundle-service
resource: /bundles/authz.tar.gz
persist: true
polling:
min_delay_seconds: 30
max_delay_seconds: 60
This configuration polls for updates every 30-60 seconds. When a new bundle activates, OPA replaces only the changed rules and data without restarting the process. Decision logs are enabled by adding a decision_logs section in the config that forwards every authorization decision to stdout or an external collector. Each log entry contains timestamp, input, result, and a unique decision id that can be correlated with application logs.
In practice this separation means application code only needs to ask a yes/no question or request a structured decision while OPA centralizes all policy logic, making audits straightforward and updates independent of application redeploys.
Rego Syntax Essentials for Authorization
Rego uses a logic programming style where rules define what is true. Every file starts with a package declaration that sets the namespace for queries. The default keyword sets a fallback value if no other rule matches.
Core constructs include:
allow { conditions }— a rule that succeeds when all conditions are true.some var in collection— existential quantification to iterate collections.input.method == "GET"— direct comparison against the query input.data.roles— reference to static or dynamically loaded data.
Here is a progressively built example that checks both static roles and dynamic ownership:
package authz.v1
default allow = false
# Helper rule for role membership
has_role(role) {
some i
data.user_roles[input.user.id][i] == role
}
# Ownership check
is_owner {
input.resource.owner == input.user.id
}
allow {
has_role("viewer")
input.method == "GET"
is_owner
}
allow {
has_role("editor")
input.method == "POST"
}
Each line is evaluated top to bottom within a rule. If any expression fails, the entire rule fails. The is_owner helper can be reused across multiple rules, reducing duplication. Built-in functions such as regex.match("^/api/v1/.*", input.path), count(input.user.groups) > 0, and time.parse_ns for time-based policies remove the need to write custom helper functions for most common cases. Version 0.68 improved error reporting so that a failed policy now shows the exact expression that evaluated to false along with the bound variable values.
Implementing Role-Based Access Control (RBAC)
RBAC policies map users to roles and roles to permissions. Store the mapping in a data.json file loaded by OPA:
{
"user_roles": {
"alice": ["admin"],
"bob": ["editor"],
"carol": ["viewer"]
},
"role_permissions": {
"admin": ["read", "write", "delete", "manage"],
"editor": ["read", "write"],
"viewer": ["read"]
}
}
The corresponding Rego policy becomes:
package rbac
default allow = false
allow {
user_roles := data.user_roles[input.user.id]
some role in user_roles
required_permission := input.permission
data.role_permissions[role][required_permission]
}
allow {
# Wildcard permission for super-admins
data.user_roles[input.user.id][_] == "superadmin"
}
To test locally, create an input.json file and run opa eval --data policy.rego --data data.json --input input.json 'data.rbac.allow'. The command returns true for allowed requests. In CI pipelines this exact command is used inside a GitHub Action or GitLab job to validate that pull requests do not break existing permissions. A production extension adds a hierarchy: an admin role implicitly includes all editor permissions using a recursive helper rule that walks the role_hierarchy object until the required permission is found or the root is reached.
This approach centralizes all role changes in one data file that can be updated by an identity provider webhook without touching application code.
Implementing Attribute-Based Access Control (ABAC) with Context
ABAC evaluates dynamic attributes such as department, clearance level, location, and resource sensitivity. A realistic input object sent to OPA contains four sections: subject, action, resource, and context.
{
"subject": {
"id": "alice",
"department": "finance",
"clearance": 3,
"location": "internal"
},
"action": "update",
"resource": {
"id": "report-123",
"owner": "alice",
"department": "finance",
"classification": "confidential",
"required_clearance": 2
},
"context": {
"time": "2024-10-01T14:30:00Z",
"ip": "10.0.0.5"
}
}
The policy that consumes this input uses multiple independent rules so that any matching condition grants access:
package abac
default allow = false
allow {
input.action == "read"
input.subject.department == input.resource.department
}
allow {
input.action == "update"
input.subject.id == input.resource.owner
input.subject.clearance >= input.resource.required_clearance
}
allow {
input.context.ip == "10.0.0.0/8"
regex.match("^internal-.*", input.subject.location)
input.resource.classification != "restricted"
}
Each rule is evaluated independently; the first match short-circuits. For performance, OPA indexes objects used in equality checks automatically. When the dataset grows to tens of thousands of resources, move static classification data into data.classifications and reference it instead of embedding inside every input object. This reduces request payload size from 2 KB to under 400 bytes per request at 500 req/s.
Setting Up OPA Locally and in Docker for Development
Begin by installing the OPA binary from GitHub releases or via Homebrew with brew install opa. Start the server with opa run --server --addr=localhost:8181 --log-level=debug. This enables detailed traces for every decision. To load initial data and policies use:
curl -X PUT --data-binary @data.json http://localhost:8181/v1/data
curl -X PUT --data-binary @policy.rego http://localhost:8181/v1/policies/authz
For containerized development the Dockerfile copies policies and data into /policies and runs opa run --server --bundle /policies. The debug log level shows every rule evaluation step, making it possible to see exactly why a request was denied. Once the policy stabilizes, switch to --log-level=info to reduce noise. This local setup mirrors production exactly, eliminating the “it works on my machine” class of bugs.
Node.js Integration Using the OPA HTTP API
A production-ready TypeScript client includes retry logic, circuit breaking, and timeout handling. The core function uses the built-in fetch with an agent for keep-alive:
import fetch from 'node-fetch';
import https from 'https';
const agent = new https.Agent({ keepAlive: true });
export async function checkAuthz(input: any, path = 'authz/allow'): Promise {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 15);
try {
const response = await fetch(`http://opa-svc:8181/v1/data/${path}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ input }),
agent,
signal: controller.signal,
});
if (!response.ok) throw new Error(`OPA returned ${response.status}`);
const { result } = await response.json();
return result === true;
} catch (err) {
// Fail open or closed depending on policy; default closed for security
console.error('OPA decision error', err);
return false;
} finally {
clearTimeout(timeout);
}
}
Use this in an Express middleware that also populates res.locals.resource from a database lookup before calling the authorization check. At 500 requests per second the observed p95 latency was 2.1 ms on a colocated OPA instance and 4.7 ms when OPA ran in a separate pod. The difference highlights the value of network locality for authorization calls.
FastAPI Integration and Dependency Injection
In Python services the opa-client library simplifies integration. A reusable dependency that accepts a required permission looks like this:
from functools import partial
from fastapi import Depends, HTTPException, Request
from opa_client import OPAClient
opa_client = OPAClient("http://opa:8181", timeout=0.010)
def require_permission(permission: str):
async def dependency(request: Request):
user = request.state.user
resource = request.state.resource
allowed = await opa_client.check(
input={
"subject": user,
"action": request.method,
"resource": resource,
"permission": permission
},
policy_path="abac/allow"
)
if not allowed:
raise HTTPException(status_code=403, detail="Insufficient permissions")
return True
return dependency
@app.get("/reports/{report_id}")
async def get_report(report_id: int, _: bool = Depends(require_permission("read"))):
return {"report_id": report_id}
The request.state objects are populated by earlier middleware that loads the user from JWT and the resource metadata from cache or database. This composable pattern lets developers declare required permissions at the route level while keeping all policy logic inside Rego.
Envoy External Authorization Filter
Envoy’s ext_authz filter offloads authorization to OPA without changing application code. The configuration uses a gRPC or HTTP service reference to the OPA sidecar. When using HTTP the filter forwards selected headers and the full request path. The Rego policy then inspects input.attributes.request.http.headers and input.attributes.request.http.path.
A complete policy for Envoy includes header parsing for JWT extraction using the io.jwt.decode built-in, role lookup, and path-based matching with glob patterns via glob.match. Production clusters run three OPA replicas per availability zone with a Kubernetes HorizontalPodAutoscaler targeting 70 % CPU. Observed added latency at the 99th percentile was 1.2 ms when decisions hit the in-memory cache and 3.8 ms on cache misses for policies containing two JWT signature validations.
Partial Evaluation for Low-Latency Decisions
Partial evaluation compiles a policy with some inputs known in advance, producing a new, smaller policy that can be evaluated quickly at the edge. The command to generate a partial policy is:
opa eval --partial --data policy.rego \
--query 'data.authz.allow' \
--unknowns '["input.resource.id", "input.user.id"]' \
--format json > partial_policy.json
The output is a new Rego document that no longer needs the full user or resource dataset. Teams push this partial policy to CDN edge workers or embed it as WASM in browser-based admin consoles. In measured workloads the partial policy reduced authorization latency from 4 ms to 0.4 ms, enabling sub-millisecond checks inside high-frequency trading or real-time bidding systems. The trade-off is that any change to the known inputs requires regeneration and redistribution of the partial policy. Automate this with a CI pipeline that runs on every merge to the main branch.
Testing Rego Policies at Scale
OPA ships with a built-in test runner. Test files live alongside policies and are named *_test.rego. A comprehensive test suite includes both positive and negative cases plus edge conditions such as missing attributes:
package authz_test
test_admin_can_delete_report {
allow with input as {
"user": {"id": "alice", "roles": ["admin"]},
"permission": "delete",
"resource": {"type": "report"}
}
}
test_viewer_cannot_delete_report {
not allow with input as {
"user": {"id": "bob", "roles": ["viewer"]},
"permission": "delete",
"resource": {"type": "report"}
}
}
test_missing_user_id_denied {
not allow with input as {
"permission": "read"
# user object deliberately omitted
}
}
Run the suite with opa test --coverage --verbose ./policies. The coverage report lists every line and expression executed. Production teams require >95 % coverage on authorization packages before merging. Replay testing loads real decision logs captured from production, replays them against a new policy version, and asserts that outcomes remain identical. This technique caught subtle bugs that unit tests missed when new built-in functions changed evaluation order.
Decision Logging, Masking, Observability and Compliance
Decision logs contain complete input objects and must be sanitized. A system.log policy defines redaction rules:
package system.log
mask["/input/subject/sensitive_field"] = true
mask["/input/resource/owner_email"] = true
# Remove entire sections for PCI data
mask["/input/context/payment_details"] = true
OPA replaces masked values with "[REDACTED]" before emitting the log. These logs are ingested by Prometheus for metrics (decisions per second, denial rate, average latency) and by Loki for full-text search during incident response. The /metrics endpoint on OPA exposes opa_server_queries and opa_server_handle_duration_seconds histograms that feed Grafana dashboards. Setting alerts on denial rate spikes above 5 % or latency p99 above 10 ms gives operators early warning of policy or data problems.
Measured Performance, Scaling Limits and Trade-offs
On a single c6i.xlarge instance (4 vCPU, 8 GB RAM) running OPA 0.68.0, a policy set containing both the RBAC and ABAC rules above sustained 1850 requests per second at < 60 % CPU with p99 latency of 4.1 ms. When the in-memory data set grew to 120 000 user-role mappings, memory usage reached 180 MB and latency increased to 6.8 ms p99. Adding three additional replicas and a simple round-robin load balancer restored sub-5 ms performance at 5200 requests per second aggregate.
Colocating the OPA sidecar with the application pod reduces network hops and brings p99 latency to 1.4 ms. The WASM module compiled from the same policy runs inside Node.js or browser runtimes with 0.6 ms median latency but requires careful memory management because each WASM instance holds its own data copy. The primary trade-off is operational complexity: sidecar deployments add another container to manage but provide language neutrality and centralized logging. Embedded WASM reduces latency at the cost of duplicating data loading logic in every runtime.
Compared with in-process libraries such as Casbin, OPA adds 1–3 ms but supplies decision logs, partial evaluation, bundle distribution, and a declarative policy language that security and compliance teams can review without reading application source code. For most startups and mid-size engineering organizations the centralized policy store outweighs the modest latency cost.
Common Production Pitfalls and Mitigations
Policy bloat is the most frequent issue. A single 1500-line Rego file becomes unmaintainable. Split by concern into rbac.rego, abac.rego, geo_fence.rego and time_window.rego, then import them with import data.rbac. Another pitfall is unbounded recursion; OPA detects cycles but the error message can be cryptic. Limit recursive helper rules to a maximum depth of three.
Stale bundle data appears when polling fails silently. Add a readiness probe that queries http://localhost:8181/health?bundles=authz and only marks the pod ready once the bundle revision matches the expected value. Finally, version bundles with semantic tags inside the manifest and keep the last four revisions available so that canary traffic can be routed to pods running the new policy while the majority fleet stays on the previous stable version.
Following these patterns—modular Rego packages, comprehensive test coverage, partial evaluation for edge use cases, colocated sidecars or embedded WASM chosen according to latency budgets, redacted decision logs, and Prometheus-backed observability—teams replace scattered hardcoded authorization checks with a single source of truth that can be audited, versioned, and updated independently of application release cycles while meeting production performance targets of under 5 ms added latency at thousands of requests per second.
