214: "Accuracy Through Repetition: When Should You Let AI Work More Independently?" ft. Justin Coats

Erik and Justin tackle viewer-submitted AI questions in a practical Q&A focused on trust, compliance, meeting recording boundaries, and how to scale an AI pilot without losing control.

🧭 Conversation Highlights

• Trust AI at work by treating it like human trust: start slow, use tasks you can verify, and measure over repetition.
• For compliance constrained data, begin with low-risk admin work, then use governance, vendor requirements, and existing industry frameworks to guide decisions.
• Decide which meetings to record by setting expectations, understanding legal/consent needs, and only recording when there is a defensible reason to keep it.
• Scale an AI pilot when measured outcomes hit agreed benchmarks, including ROI and intangibles like morale and “intelligence debt.”

💡 Key Takeaways

• Trust is earned over time. Use oversight at first, then loosen it only as the system proves itself repeatedly.
• If data is sensitive, don’t start with frontier use cases. Start with safe tasks and align adoption to org policy and vendor compliance requirements.
• Recording is not default permission. Socialize expectations, declare the recording approach, and be ready to stop if the topic crosses privacy lines.
• Scaling is not about vibes. Define pilot success metrics up front, then compare benchmark results against your trade-offs. Don’t confuse speed with accuracy. Measure both where you can.

❓ Questions That Mattered

• How do I know when to trust AI at work versus slow down and check it?
• If I work with protected data, what are the first steps to use AI safely and compliantly?
• Which meetings should be recorded by default, and where are the privacy boundaries?
• What should an AI pilot prove before a company scales it, and when should it stop? (Also: how do we choose the right tools and train people as they change?)

🗣️ Notable Quotes

• Trust with humans is built over time, so with AI you should start slow too.
• Accuracy through repetition moves your trust level up, especially when you have a rubric or rule set to measure against.
• If it’s not clear why you’re recording, why are you doing it?
• You might have a massive increase in employee morale and happiness, but the ROI might not match right away, and that trade-off matters.

🔗 Links & Resources

• Listen To Other Episodes Co-Hosted With Justin