Devin Huber

AI Automation & Agentic Systems LLM Applications, RAG & AI Evaluation

Columbus, OHD.Huber.Professional@gmail.comdevinhuber.comlinkedin.com/in/devinhubergithub.com/DHuberPortfolio

Professional Summary

AI solutions analyst who builds AI agents and LLM applications for business operations and measures how often they are wrong. Created an organization’s AI function and runs its production workflows in n8n, Zapier and Claude. Builds with Claude Code in Python and Node.js: a retrieval-augmented (RAG) assistant served over MCP on Google Cloud Run, a financial-statement spreader with accounting tie-outs, and an open-source evaluation harness that exposed a 27% silent error rate in a spec-build agent’s stress test and verified the fix (8.6%, worst of three runs).

Technical Skills

AI & LLM: Claude (Opus, Sonnet, Haiku), Anthropic API, Claude Agent SDK, Claude Code, Model Context Protocol (MCP), prompt engineering, structured outputs, tool use via MCP, retrieval-augmented generation (RAG: BM25, embeddings), AI agents and agentic workflows, groundedness checks, guardrails, human-in-the-loop review, prompt-injection defense
Automation & Integration: n8n, Zapier (incl. Agents), low-code/no-code workflow orchestration, REST APIs, webhooks, JSON, Microsoft Graph API, Google Analytics 4, Kajabi, Google Sheets, Claude scheduled tasks, Claude Cowork
Evaluation & Testing: LLM evaluation (evals), hallucination detection, evaluation sets (golden sets / answer keys), silent error rate, precision / recall / F1, Wilson confidence intervals, held-out testing, run-to-run variance, regression testing, CI release gates, cost per unit
Languages, Frameworks & Cloud (built with Claude Code): Python, JavaScript / Node.js, TypeScript, FastAPI, React, pytest, ONNX Runtime, SQLite, Git, GitHub Actions CI, Docker, Google Cloud Run, Secret Manager, Netlify

Professional Experience

AI Solutions AnalystOct 2025 – Present

Innovative Leadership Institute, Columbus, OH

Created the organization’s AI function and run its production AI workflows; moved automations from Zapier to n8n, with Claude orchestrating and no-AI n8n workflows holding allow-lists and fixed recipients so every write stays deterministic.
Built an AI email assistant, live since Oct 2026, for the CEO’s and shared inquiries mailboxes (n8n, Claude Sonnet, Microsoft Graph API) that sends a reply on its own only inside limits enforced in code (confidence and importance thresholds, VIP holds, link and recipient checks, kill switch) and puts everything else in a nightly to-do email the CEO approves by reply; 97 tests.
Rebuilt a two-agent guest-application pipeline (Zapier, then a Claude scheduled task over n8n) that scores applicants on a rubric, checks each against the CEO’s mailbox, contacts and an exclusion list before any decline goes out, and sends a monthly digest; cleared a 21-application backlog (~110 hours/year projected back to the CEO).
Transcribed all 343 podcast episodes with AI-assisted transcription and indexed the archive; repaired 38 of 153 audited transcripts at the source, and built a weekly agent that drafts a newsletter section around a transcript-verified quote.
Released the Podcast Archive Assistant to colleagues in Claude Cowork (plugin, skills, user guide); first point of contact for generative AI, turning recurring questions into prompt templates, SOPs and knowledge-base content.

Projects

Code: github.com/DHuberPortfolio. Write-ups: devinhuber.com. Results are scored against hand-checked answer keys.

Podcast Archive Assistant (Python, MCP, RAG, Google Cloud Run; released to colleagues). Python MCP server (14 tools) giving Claude BM25 and vector-embedding retrieval over 343 episodes; code verifies every quote and routes each answer to ANSWERED, FLAGGED or DECLINED. On a 161-question answer key: 157 routed correctly, 0 of 33 false-premise or out-of-scope questions answered as true, 4.7% silent error rate. Cloud Run (Docker, Secret Manager); ONNX cut cold start 46 s to 8.5 s; 183 tests in CI.

Financial Statement Spreader (Python, FastAPI, React/TypeScript; private, demo on request). Spreads SEC 10-K statements into a standard credit template with 19 accounting tie-out checks, credit ratios and the source of every value; only rules with measured accuracy auto-accept, the rest go to an exception queue. On a 30-filing holdout scored against inline-XBRL answer keys: 81.6% of values correct, 1.5% of auto-accepted values wrong. 1,229 tests; CI release gate.

Workflow Evaluation Harness (Node.js CLI, open source). Zero-dependency CLI that grades any AI workflow’s output against an answer key: silent error rate with Wilson intervals, review precision, calibration, and variance/compare/what-if commands that separate real gains from noise. Two 500-record golden sets, 17 runs scored, 217 tests in CI.

Spec builds for prospective employers (n8n, scored with the harness). Metadata-tagging agent: a 500-article stress test exposed a 27% silent error rate that 24 test articles had missed; a revised prompt and a guard against invented codes cut it to 8.6% (worst of three runs). Bar-compliance reviewer: its scorecard reported a 100% catch rate; 30% of the copy it let through still needed a person. An anchored-confidence prompt cut that to 6.7% in four of four runs.

Education

Bachelor of Science in Business Administration, Finance 2025
The Ohio State University, Fisher College of Business, Columbus, OH Minor: History