Building Generative AI Systems that Ship
What I learned building RAG applications in production: why retrieval quality matters more than model size, how to evaluate without ground truth, and why prompt engineering is really interface design.
Notes
Notes on AI, cloud & data engineering
Notes and write-ups on things I've built, broken, and learned along the way.
Cloud Architecture: Lessons from OCI and Azure
What I learned building secure cloud infrastructure across OCI and Azure: why Infrastructure as Code is non-negotiable, why private endpoints matter, and why CI/CD is the only way to keep cloud costs predictable.
Machine Learning Systems Design
Building ML systems that work in production: data drift, model monitoring, feature store trade-offs, and why most ML failures happen outside the model itself.
Building Generative AI Systems that Ship
Start with retrieval, not generation
Most teams I talk to start with the model: which LLM to use, how to fine-tune it, what prompt to write. But for most enterprise use cases, the bottleneck isn't the model — it's retrieval. If your retrieval surface returns irrelevant documents, no LLM in the world will give a good answer.
I've found that spending time on chunking strategy, embedding quality, and re-ranking produces bigger wins than swapping models. A mediocre model with great retrieval beats a great model with bad retrieval every time.
Evaluation is the hard part
Without ground truth labels, how do you know if your RAG system is getting better? I've found LLM-as-judge evaluation, sentence-level similarity checks, and golden datasets to be the most practical approach. The key is having some automated signal you can trust, even if it's imperfect.
Prompt engineering is interface design
Treating prompts as configuration rather than code makes them versionable, testable, and reviewable. I keep prompts in version control, version them alongside code, and treat prompt changes as code changes that need review.
Cloud Architecture: Lessons from OCI and Azure
Infrastructure as Code is non-negotiable
Any cloud resource that isn't defined in code is technical debt. I've seen too many "temporary" manual changes become permanent bottlenecks. Terraform (or Pulumi, or Pulumi/CDK) makes infrastructure reviewable, versionable, and reproducible.
Private endpoints everywhere
Exposing databases and storage to the public internet "for convenience" is a ticking time bomb. Private endpoints, VNet isolation, and Key Vault for secrets should be the default, not the exception.
CI/CD is the only way to manage cloud cost
Untracked resources are how cloud bills spiral. Automated deployment, tagging policies, and regular cost reviews keep infrastructure spend predictable rather than surprising.
Machine Learning Systems Design
Most ML failures happen outside the model
Data drift, feature skew, training/serving mismatch, and bad monitoring cause more production failures than model accuracy ever does. A model with 95% accuracy that drifts silently is worse than a model with 85% accuracy and good monitoring.
Data quality > model size
Clean, well-labeled data beats a bigger model trained on messy data. Feature engineering and data quality checks are higher-leverage than model architecture tweaks for most production use cases.
Monitor the pipeline, not just the model
Input distribution, feature completeness, latency, and error rates matter as much as model accuracy. Alert on data quality, not just model performance.
These notes are living documents — I update them as I learn more. If you have thoughts or corrections, feel free to reach out via the contact page.