Route Inference Through a Gateway

Introduction

In the Workers AI course, an application sent a prompt directly to a Cloudflare-hosted model. That works, but a growing application also needs one consistent place to observe and control model traffic. Cloudflare AI Gateway is that checkpoint: the caller sends a request to a named gateway, and the gateway forwards the request to an upstream model provider such as Workers AI.

This lab keeps the three roles visible:

• the caller is curl in your LabEx VM;
• the gateway checks whether the caller may enter and records the request;
• the upstream provider is Workers AI, which checks whether the request may run the model.

Those last two checks use separate credentials. cf-aig-authorization authenticates the caller to AI Gateway. The ordinary Authorization header authenticates the gateway request to Workers AI. A valid gateway token is not automatically a Workers AI credential, and a Workers AI credential does not bypass an authenticated gateway.

You will create one disposable authenticated gateway in the Cloudflare Dashboard, create a narrowly scoped AI Gateway token, send a short request to the Cloudflare-hosted Llama 3.3 model and inspect the resulting log. You will then replace only the gateway credential with an invalid value to prove which boundary rejects the request. Finally, you will delete the gateway, delete its token so it can no longer authorize requests, and log Wrangler out.

If you entered this course directly, first complete Connect LabEx to Your Cloudflare Account. It teaches the LabEx VM terminal, Wrangler device authorization, account confirmation and explicit account IDs. The Workers AI inference lab is also a useful prerequisite.

AI Gateway is available on the Free plan, and its core logging is free within the account limits. The selected @cf/meta/llama-3.3-70b-instruct-fp8-fast model can use the shared Workers AI free allocation with Standard billing. Workers Paid and Unified Billing are not required. Stop instead of repeatedly retrying if the account's daily Workers AI allocation is unavailable.

Setup installs Node.js 22.22.0 and project-local Wrangler 4.132.0 in /home/labex/project/ai-gateway-route. It provides independent read-only assessments, but it does not authorize Wrangler, create a token or gateway, send inference, or modify your Cloudflare account. LabEx does not save this temporary VM after the lab ends. You will still delete the cloud token and erase its VM copy explicitly so cleanup is complete before the VM is destroyed.

Authorize the VM and Record Owned Names

In this step, you will connect the fresh VM to your learning account and save names that make this lab's resources unambiguous.

Wrangler's device login authorizes Workers AI, but it does not create the separate AI Gateway caller credential used later. Keeping those credentials distinct makes the trust boundary easier to see.

Enter the prepared project and confirm the pinned CLI version:

cd /home/labex/project/ai-gateway-route
npx wrangler --version

Expect 4.132.0. Start device authorization with account identity and Workers AI access:

npx wrangler login --device --browser=false --scopes account:read user:read ai:write

Open the displayed link, enter the code and authorize the intended learning account. Then inspect structured identity:

npx wrangler whoami --json

Confirm loggedIn: true. Create a unique gateway ID and its related token name. Replace YOUR_ACCOUNT_ID with the actual 32-character ID shown for the intended account:

GATEWAY_ID="labex-c09-g01-$(openssl rand -hex 6)"
TOKEN_NAME="$GATEWAY_ID-token"
cat > .labex/state.json <<JSON
{
  "accountId": "YOUR_ACCOUNT_ID",
  "gatewayId": "$GATEWAY_ID",
  "tokenName": "$TOKEN_NAME"
}
JSON
cat .labex/state.json

The random suffix prevents collisions. The state file contains resource identifiers, not credentials, and lets every later command target the exact resource this lab owns.

Create an Authenticated Gateway and Caller Token

In this step, you will create the checkpoint and a credential that can both call and inspect it.

Open the Cloudflare Dashboard and choose AI → AI Gateway → Create gateway → Custom gateway. Use the gatewayId from .labex/state.json as the gateway name. Keep these settings:

• request logging: on;
• gateway authentication: on;
• cache, rate limits, spend limits and retries: off;
• Workers AI billing: Standard.

Standard billing keeps Workers AI usage in its normal allocation. Unified Billing is a different payment path and is outside this beginner lab.

After Cloudflare opens the new resource, use the breadcrumb and selected Overview tab to confirm that you are inside the exact disposable gateway rather than the account-wide gateway list.

After creation, open Settings. Confirm that the displayed gateway ID exactly matches your saved ID and that logging and authentication are enabled.

Now choose Create an AI Gateway authentication token. Name it with the saved tokenName, select only the intended learning account, and add these permissions:

• AI Gateway — Run lets the caller enter an authenticated gateway;
• AI Gateway — Edit lets the lab read and delete AI Gateway resources through the management API.

Do not add Workers AI permission to this token. Workers AI remains authorized by Wrangler's separate short-lived credential.

Create the token only after reviewing the account and permissions. Cloudflare shows its value once. Store it privately without echoing it:

bash -c '
while :; do
  read -rsp "Paste the AI Gateway token: " GATEWAY_TOKEN
  printf "\n"
  [ -n "$GATEWAY_TOKEN" ] && break
  printf "Token cannot be empty; paste it again.\n" >&2
done
umask 077
printf "%s" "$GATEWAY_TOKEN" > .labex/gateway-token
unset GATEWAY_TOKEN
chmod 600 .labex/gateway-token
'

The prepared terminal uses zsh interactively, so this block starts a short Bash subprocess for Bash's hidden read prompt. An empty paste is rejected before the command can return to the shell. The token stays only in the subprocess and the private file.

The token is deliberately kept outside configuration and command output. Verify the real gateway through the authenticated management API:

ACCOUNT_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).accountId')
GATEWAY_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).gatewayId')
GATEWAY_TOKEN=$(cat .labex/gateway-token)
curl --http1.1 -fsS \
  -H "Authorization: Bearer $GATEWAY_TOKEN" \
  "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways/$GATEWAY_ID" \
  | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{const b=JSON.parse(s),g=b.result||{};console.log(JSON.stringify({success:b.success,id:g.id,collect_logs:g.collect_logs,authentication:g.authentication},null,2))})'
unset GATEWAY_TOKEN

Expect the owned ID with both collect_logs and authentication set to true. No secret is printed.

Route One Workers AI Request Through the Gateway

In this step, you will send one small request through the gateway instead of directly to Workers AI.

The provider-native gateway URL contains the account, gateway, provider and model. The two authorization headers deliberately remain separate:

caller → cf-aig-authorization → AI Gateway → Authorization → Workers AI model

Obtain Wrangler's current short-lived Workers AI token as structured data, then make the request. The command writes only the JSON response to disk; neither credential is printed:

ACCOUNT_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).accountId')
GATEWAY_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).gatewayId')
MODEL='@cf/meta/llama-3.3-70b-instruct-fp8-fast'
GATEWAY_TOKEN=$(cat .labex/gateway-token)
UPSTREAM_TOKEN=$(npx wrangler auth token --json | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write(JSON.parse(s).token))')
curl --http1.1 -fsS \
  -H "cf-aig-authorization: Bearer $GATEWAY_TOKEN" \
  -H "Authorization: Bearer $UPSTREAM_TOKEN" \
  -H 'Content-Type: application/json' \
  --data '{"prompt":"In one sentence, explain why an AI gateway is useful.","max_tokens":64}' \
  "https://gateway.ai.cloudflare.com/v1/$ACCOUNT_ID/$GATEWAY_ID/workers-ai/$MODEL" \
  > .labex/valid-response.json
unset GATEWAY_TOKEN UPSTREAM_TOKEN
node -e 'const b=require("./.labex/valid-response.json"); console.log(b.result?.response ?? b.result)'

The wording may differ because generation is nondeterministic. The assessment checks only that the provider returned a successful nonempty result through the owned gateway.

Isolate the Gateway Authentication Boundary

In this step, you will keep the valid Workers AI credential but replace only the gateway credential.

A controlled negative test should change one condition at a time. If both credentials were invalid, an HTTP failure would not tell you which system rejected the request. This request retains Wrangler's valid upstream token and sends a clearly invalid cf-aig-authorization value:

ACCOUNT_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).accountId')
GATEWAY_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).gatewayId')
MODEL='@cf/meta/llama-3.3-70b-instruct-fp8-fast'
UPSTREAM_TOKEN=$(npx wrangler auth token --json | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>process.stdout.write(JSON.parse(s).token))')
STATUS=$(curl --http1.1 -sS -o .labex/invalid-response.json -w '%{http_code}' \
  -H 'cf-aig-authorization: Bearer deliberately-invalid' \
  -H "Authorization: Bearer $UPSTREAM_TOKEN" \
  -H 'Content-Type: application/json' \
  --data '{"prompt":"This request must not reach the model.","max_tokens":8}' \
  "https://gateway.ai.cloudflare.com/v1/$ACCOUNT_ID/$GATEWAY_ID/workers-ai/$MODEL")
unset UPSTREAM_TOKEN
printf '%s\n' "$STATUS" | tee .labex/invalid-status.txt

Expect 401 or 403. Do not display the response body: the status is sufficient evidence, and keeping error output bounded reduces the chance of exposing request details.

Connect the Request to Its Gateway Log

In this step, you will use observability to connect runtime behavior with a visible gateway record.

Observability means collecting enough evidence to explain what a system did after a request left the caller. A gateway log can show the provider, model, status, latency and token usage without asking the model again. Logs may take a short time to appear.

Read the existing logs through the authenticated management API. This is a read-only check; it does not send another model request:

ACCOUNT_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).accountId')
GATEWAY_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).gatewayId')
GATEWAY_TOKEN=$(cat .labex/gateway-token)
curl --http1.1 -fsS \
  -H "Authorization: Bearer $GATEWAY_TOKEN" \
  "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways/$GATEWAY_ID/logs?per_page=50" \
  > .labex/logs.json
unset GATEWAY_TOKEN
node - <<'NODE'
const body = require('./.labex/logs.json')
const model = '@cf/meta/llama-3.3-70b-instruct-fp8-fast'
const matches = (body.result || []).filter(row =>
  row.provider === 'workers-ai' && row.model === model
)
console.log(matches.map(row => ({
  id: row.id,
  provider: row.provider,
  model: row.model,
  success: row.success,
  created_at: row.created_at
})))
if (!matches.some(row => row.success === true)) process.exit(2)
NODE

Expect one entry with provider: "workers-ai", the intended model and success: true. If the command exits without that entry, wait about 20 seconds and rerun this same read-only block rather than sending more inference requests.

Open the gateway's Logs view in the Dashboard. Find the successful Workers AI row for @cf/meta/llama-3.3-70b-instruct-fp8-fast. Confirm success, provider and model before opening its detail panel.

The exact duration, token count and generated text can vary. Those values describe this request; they are not targets to reproduce exactly. Never place credentials or personal information in a prompt merely to make a log easier to find.

Delete the Disposable Gateway

In this step, you will remove the cloud resource while its management credential is still available.

Cleanup must target the exact owned ID and must be proved by an authenticated inventory. A missing page caused by logout or a network failure is not proof of deletion.

ACCOUNT_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).accountId')
GATEWAY_ID=$(node -p 'JSON.parse(require("fs").readFileSync(".labex/state.json")).gatewayId')
GATEWAY_TOKEN=$(cat .labex/gateway-token)
curl --http1.1 -fsS -X DELETE \
  -H "Authorization: Bearer $GATEWAY_TOKEN" \
  "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways/$GATEWAY_ID" \
  | node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{const b=JSON.parse(s);if(!b.success)process.exit(1);console.log("gateway deletion accepted")})'
unset GATEWAY_TOKEN

GATEWAY_TOKEN=$(cat .labex/gateway-token)
curl --http1.1 -fsS \
  -H "Authorization: Bearer $GATEWAY_TOKEN" \
  "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai-gateway/gateways" \
  > .labex/gateways-after-delete.json
unset GATEWAY_TOKEN
node -e 'const b=require("./.labex/gateways-after-delete.json"),id=process.argv[1],found=(b.result||[]).some(g=>g.id===id);console.log("gateway absent:",!found);if(found)process.exit(1)' "$GATEWAY_ID"

Expect gateway absent: true. This second request lists gateways with valid authorization and fails if the owned ID remains. Other gateways in your account are never modified.

Delete the Token and Log Out

In this step, you will remove the two independent credentials in the reverse order from which you used them.

In the Cloudflare Dashboard, open My Profile → API Tokens. Find the exact token name stored in .labex/state.json, open its Actions menu, choose Delete, inspect the confirmation and delete only that token. Deleting the token revokes its access immediately. It is safe to remove now because the gateway is already gone.

Remove its local copy, then end Wrangler's separate VM authorization:

shred -u .labex/gateway-token
npx wrangler logout
npx wrangler whoami --json || true

Expect structured output with loggedIn: false. The Dashboard browser session is separate and remains signed in. Run the final local check:

test ! -e .labex/gateway-token && echo "local gateway token removed"

The message confirms that the VM copy is absent. The LabEx Check button independently repeats the local file and Wrangler logout checks; its backend script is intentionally not part of the learner project.

You have now removed the gateway, deleted its caller/management token, erased the local token copy and disconnected the fresh VM. When you end the lab, LabEx destroys this temporary VM instead of saving it; cloud cleanup still matters because destroying a VM alone cannot revoke a Cloudflare token or remove a gateway.

Summary

You created an authenticated Cloudflare AI Gateway and routed a real Workers AI inference through it. You kept gateway authorization separate from upstream model authorization, changed only one credential to identify the rejecting boundary, and connected the successful request to its gateway log. Finally, you proved authenticated resource deletion before deleting the token and logging the VM out.

The next lab builds on this observable request path. You will attach small nonsecret metadata, trace a deliberate failure and use evidence from the gateway rather than guessing where a request failed.