Justin

Command Palette

Search for a command to run...

Back to project archive
Completed2026ai-data

LLM Train-Eval-Ship Release Console

A guided LLM MLOps journey for release gates, evidence, canary rollout, and rollback

A mock-safe guided product journey that turns the LLM train → eval → ship path into an interactive release console. Users can operate release candidates, inspect checkpoint evidence, review AutoEval scorecards, and decide whether to promote, hold, or roll back an adapter without GPUs, model weights, API keys, or external services.

Live DemoView GitHubDemo RecordingRead README

Verified Demo Recording 1

Copied by the portfolio quality pass from existing project demo media.

29 screenshots4 demo videos

Media overview

Browse the project screenshots and recorded walkthroughs.

Project links and demo readiness

live

Live Demo

Open the verified live demo when available, or use the internal run guide for manual demo preparation.

github

View GitHub

Open the public repository when available, or review the source-access status captured by this case study.

video

Demo Recording

Open the demo recording when available, or use this section as the recording checklist for the project.

documentation

Read README

Open the project README when available, or use this case study as the documentation baseline.

Project overview

## Overview LLM Train-Eval-Ship does not pretend to fine-tune a large model in the browser. Its purpose is to make the most important release decisions visible: whether data is ready, whether an adapter artifact is registered, whether evaluation passed, whether users should see the canary, and how rollback happens when regression appears. ## What is implemented The FastAPI backend exposes a mock-safe API contract for health checks, demo scenarios, pipeline runs, AutoEval scorecards, and deployment manifests. The frontend is a GitHub Pages-ready guided release console with three switchable scenarios. The README is English-only and includes Mermaid flow, architecture, data flow, deployment, module organization, and tech-stack diagrams. ## How to use it Open the interactive demo, switch between Support Copilot LoRA, Policy DPO Safety Pass, and RAG Agent Regression Gate, then follow the checkpoint lane, scorecard, user decision panel, and operation log. The video and screenshot gallery preserve the same workflow states, while the README documents local startup, tests, deployment, API contracts, and roadmap boundaries. ## Honest boundary This version is a portfolio-ready mock-safe demo. It demonstrates pipeline contracts, release decisions, tests, capture assets, and deployment. Real model training, production vLLM/TGI serving, artifact registries, and live canary metrics remain future extension points.

My role

Solo developer (architecture, backend API, demo app, testing, capture pipeline, and deployment)

Problem

LLM release workflows normally require GPUs, large model weights, private data, and serving infrastructure. A public demo needs to let users understand the release decision path safely, not just read a README or call a health check.

Solution

I turned a thin FastAPI skeleton into a mock-safe guided release console: demo APIs, three release scenarios, AutoEval scorecards, deployment manifests, a GitHub Pages product UI, Playwright screenshots, a WebM walkthrough, and an English README with Mermaid diagrams for flow, architecture, data movement, deployment, module organization, and the technical stack.

Current outcome

Delivered a locally runnable, smoke-tested, Docker-buildable, GitHub Pages-ready LLM MLOps demo with screenshot and recording assets integrated into the main portfolio. Real LoRA/DPO training and production vLLM/TGI serving remain clearly labeled roadmap items.

Highlights

  • The first screen shows a guided release console: queue, checkpoint lane, scorecard, user decision, and operation log
  • Mock-safe mode needs no GPU, model weights, API keys, or external services
  • FastAPI exposes /healthz plus scenario, pipeline, scorecard, and manifest APIs
  • Shared model-cache governance through MODEL_STORE_ROOT, HF_HOME, TRANSFORMERS_CACHE, and HF_HUB_CACHE
  • Playwright + FFmpeg produce cover, screenshots, and WebM demo recording
  • GitHub Pages workflow plus Docker/Nginx static smoke path

Engineering challenges

  • Representing LLM MLOps decisions honestly without running a real model in the public demo
  • Turning a /healthz-only skeleton into a user-operable release journey

Target users

  • Portfolio reviewers and interviewers
  • Technical readers who need a quick view of purpose, stack, and maturity

Technical highlights

  • Detected technical signals: Python, FastAPI, Uvicorn, Vanilla JavaScript, Playwright, FFmpeg, Docker, Nginx, GitHub Actions, GitHub Pages, Hugging Face, vLLM, TGI
  • README evidence exists and can support a fuller reviewed case study
  • A public GitHub repository is linked for source traceability

Architecture

This case study is generated from the portfolio catalog pipeline using README, Git metadata, package/build configuration, and media signals. The final architecture narrative still needs source-level review. Current detected technology signals include: Python, FastAPI, Uvicorn, Vanilla JavaScript, Playwright, FFmpeg, Docker, Nginx, GitHub Actions, GitHub Pages, Hugging Face, vLLM, TGI.

Data flow

A public data-flow narrative is not fully reviewed yet. If the project includes data processing, AI pipelines, or backend APIs, the next pass should document input, processing, storage, and UI/output flow end to end.

Project structure

llm-train-eval-ship/
  README.md              # project documentation, when available
  source files           # implementation reviewed by local audit
  package/build config   # detected capability signals

Setup / Run guide

This project does not expose a verified runnable web command yet. Review the README/source tree and add exact install, run, test, and build commands before interview use.
No verified build command was detected. Treat the current portfolio page as a case-study placeholder until build steps are reviewed.

Future improvements

  • Complete the production-quality README, screenshots, and demo recording
  • Add architecture diagrams, data-flow notes, and key technical decisions
  • Verify build/test status and update the portfolio release report

Interview notes

  • State the current maturity and demonstrable scope first
  • Focus on verified stack, source structure, and completed behavior
  • Do not claim unverified deployment, video, or test coverage as finished

Next steps

  • Wire in real LoRA/PEFT or DPO job runners
  • Connect vLLM/TGI serving endpoints and real canary metrics
  • Add real eval datasets, artifact registry, and model version governance

Related projects