View/Purchase Link

Notes / Summary

A Practical Guide to Experimental Methods in Engineering

A comprehensive toolbox of techniques for evaluating new features and fine-tuning systems. This book provides practical guidance on designing, running, and analyzing experiments that help engineers make data-driven decisions in competitive industries.

What You’ll Learn

The book covers essential experimental techniques for modern engineering:

• A/B Testing: Design, run, and analyze controlled experiments effectively
• Multi-Armed Bandits: Increase experimentation rate and minimize regret
• Bayesian Optimization: Tune multiple parameters efficiently using probabilistic methods
• Feedback Loop Management: Break cycles caused by periodic ML model retraining
• Business Metrics: Clearly define and measure success criteria for decision-making
• Pitfall Avoidance: Identify and avoid common experimental design mistakes

Key Features

• Practical Focus: Real-world examples from competitive industries
• Cost Minimization: Techniques to reduce experimental costs while maximizing insights
• Python Implementation: Code examples using Python and NumPy
• Business Impact: Methods to quickly identify approaches that deliver the best results

Target Audience

Perfect for ML engineers and software engineers looking to extract maximum value from their systems through systematic experimentation. The book bridges the gap between academic experimental design and practical engineering applications.

About the Author

David Sweet brings extensive experience as a quantitative trader at GETCO and machine learning engineer at Instagram, where he applied experimental methods to tune trading systems and recommender systems. He has lectured on experimental optimization at NYU Stern and teaches in AI and Data Science master’s programs at Yeshiva University.

This book is essential for any engineer who wants to make data-driven decisions through systematic experimentation.