03 / ENGINEERING CASE STUDY
Relaywise
A no-code, AI-native workflow platform with a Next.js frontend and FastAPI backend. It uses a LangGraph ReAct loop to dynamically discover tools across 800+ Composio OAuth integrations, queueing requests through SQS FIFO and streaming live tool actions over AppSync WebSockets.
Memory & Checkpointing
Uses LangGraph AsyncPostgresSaver to checkpoint within-thread state so chats can pause and resume, AsyncPostgresStore for durable cross-session user facts, and SummarizationMiddleware to condense history past 16k tokens.Evaluations & Tracing
Built a custom evaluation harness that runs free deterministic tests for intent routing in CI, and traces expensive LLM memory extraction tests to LangSmith when tuning prompts.Audited Security Scanners
Discovered Snyk's Python scanner silently skipped 31 core packages. Python scanning was moved to pip-audit for full coverage, while Snyk still handles SAST, IaC, and npm.Token Cost Optimization
Enforces strict usage metering. Every API action is priced dynamically and deducted from a $0.10 rolling monthly budget stored in an ultra-fast Redis cache to prevent LLM abuse.PRODUCT, IN PLAIN ENGLISH
Relaywise is a workflow automation platform that carries out tasks across the SaaS apps a user connects to it. In the Next.js frontend, users can browse a catalog of 860+ apps, securely connect their own accounts via OAuth, and disconnect them at will. When a user asks a question, the backend figures out the target app, dynamically searches Composio to discover what tools exist, reads their schemas to learn what arguments they take, and executes them against the app's REST API using the user's OAuth credentials.
REQUEST / EVENT LIFECYCLE
TECHNICAL ARCHITECTURE
How it is built.
The agent relies on a ReAct loop running in LangGraph. Instead of blindly passing hundreds of tool definitions in the system prompt, it uses three specific Composio meta-tools to search for capabilities, fetch arguments, and execute actions at runtime. Before spending a single model token, a deterministic router figures out the target app from the chat history or a Redis-backed usage sorted set. Every user is strictly capped by a $0.10 monthly budget metered by actual AWS token usage.
FULL STACK / CLOUD / DEVOPS
The stack, organized
by responsibility.
Frontend & identity
Next.js 16 App Router on Vercel. Clerk issues the JWT, which is verified twice: once by a Next.js proxy to protect routes, and again by the AppSync Authorizer Lambda. Zustand manages client state (chat lists, connected apps), while shadcn/ui and Tailwind handle styling.
GraphQL & Subscriptions
The Apollo Client uses `aws-appsync-subscription-link` to split traffic: standard GraphQL queries and mutations are routed over HTTP, while live tool-execution streams are pushed over a persistent WebSocket connection.
Backend & agent runtime
FastAPI + Mangum. The agent uses LangGraph's ReAct state machine, routing dynamic Composio tools to the Bedrock Mantle minimax-m2.5 model.
Data & memory
Neon Postgres (serverless SQL) handles chat names and LangGraph state. Upstash Redis handles rate limiting via a $0.10 monthly credit allowance, tracks which apps a user has connected, and provides deterministic routing signals.
Observability & SRE
AWS Lambda Powertools injects task/session IDs into every log for easy CloudWatch Insights tracing. Metrics use the Embedded Metric Format (EMF) to print directly to stdout without blocking handlers. SRE alarms (like DLQ alerts) watch metrics emitted by AWS itself, ensuring they trigger even if a container crashes during initialization. AWS Budgets alert at 50%, 80%, and 100% usage.
AWS cloud architecture
Fully serverless via Terraform. Infrastructure is split into two independent stacks (`orchestration` for AppSync and `api` for compute) so schemas and code deploy independently. The LangGraph Agent Worker Lambda runs as a container image pulled from Amazon ECR (Elastic Container Registry) to handle heavy dependencies.
DevOps & Pipeline
A hardened GitHub Actions pipeline runs pytest, vitest, Snyk, and pip-audit. Vercel deployments are staged with --skip-domain, smoke-tested, and only then promoted to production.
ENGINEERING DECISIONS
Why these choices matter.
To handle cold starts and AWS Lambda timeouts, the request splits in two. Synchronously, the API authorizes the user, checks credits, auto-titles the conversation in Postgres, and queues a task in under a second. Asynchronously, a worker pulls the task from the SQS queue and runs the LangGraph loop. Because users want to know what the agent is doing, every single tool call (like "Reading Slack channel...") streams live to the browser over an AppSync GraphQL WebSocket subscription as it happens.