AI Implementation at Scale, Tools, Agents & Benchmarks: Jacob Miller (Pattern) | DeFi Podcast #61

In this episode of the DeFi Podcast, we're joined by Jacob Miller, VP of Data Science at Pattern, to explore how AI is being implemented at enterprise scale in one of e-commerce's biggest players.

Jacob reveals Pattern's approach to benchmarking various AI models (Claude, Gemini, GPT) against real business metrics rather than public leaderboards, and why fine-tuning is finally becoming worthwhile after years of being a "trap."

As the VP overseeing AI operations for the #1 seller on Amazon globally, Jacob offers rare insights into how commercial AI is transforming business operations and driving tangible ROI.

Alongside hosts Bill, JF, and Rhett, we dive into Jacob's contrarian take on AI agents, why different AI models are becoming increasingly specialized, and his fascinating personal applications of AI - from creating a Chinese language tutor for his son to building an AI coach that helps him stay aligned with his goals throughout the day.

Key Topics:

- How Pattern swaps and benchmarks different AI models to determine which works best for specific business tasks

- Why fine-tuning AI models is finally becoming valuable after previously being a "trap"

- Jacob's "hot take" that AI agents are currently overrated but will be valuable in ~2 years

- Creative personal AI applications including an AI coach and Chinese language tutor

- Realistic perspectives on AI development timelines and the limitations of current technology

- The intersection of AI and blockchain - why crypto may have an advantage for AI agent integration

Chapters:

00:00 Introduction and Guest Welcome

01:40 Jacob Miller's Background and Role at Pattern

04:03 AI Models and Their Applications

06:35 Public vs. Private Benchmarks in AI

10:05 Writing and AI: Personal Experiences

12:25 Consumer AI and User Experience

16:10 Google's AI Journey and Market Dynamics

19:21 Future of AI: Investment and Bottlenecks

25:19 Fine-Tuning AI Models: Is It Worth It?

28:35 Surprises and Contrarian Views in AI

29:48 Unexpected Breakthroughs in AI

29:54 The Rapid Evolution of Image Generation

31:05 The Competitive Landscape of AI Models

33:33 AI Agents: Overhyped or Future Game-Changers?

42:23 AI in Personal and Professional Life

53:08 The Future of AI and Its Potential Risks

01:01:56 Concluding Thoughts and Predictions

Follow us:

Twitter: @DefiPodcast

Guest links:

@patternhq

Follow the Hosts:

@Bill_Tanyeri

@JFSaine

@RhettShipp

Disclaimer: Nothing discussed on this podcast constitutes financial advice. The hosts and guest may hold positions in some of the assets or projects mentioned during the episode. Please consult a professional investment advisor and do your own research before making any financial decisions.