Description
An agent will hand you an analysis that runs, looks clean, and reads convincingly, and is quietly wrong in a way no error message will ever flag. A denominator that silently dropped half the users, so engagement looks up. An average dragged by one whale account, sold as the typical customer. A result that turned significant only because someone checked eight metrics and peeked every morning. A lift that reverses the moment you split it by country, because a lurking variable was doing the work. A correlation written up as a cause. An axis that starts at 80 to fake a cliff for the board. The query ran and the chart rendered, which proves the code was well formed, not that the number is true, the experiment was valid, the picture is honest, or the use of the data was legal. Tool tutorials do not help here, you are not the one typing the query. What you lack is the reviewer’s mental model: what would have to be true for this number to be wrong, and whether anyone checked, so you can look at an analysis and know whether to act on it or send it back.
This book trains you to be the reviewer and decision-maker over data, not to memorize another tool. It starts from a different premise than every other analytics book: the agent writes the queries, the pandas, and the first draft of the chart, and your value is the judgment the tool does not have. You’ll learn to frame the question behind the question a stakeholder actually asked, to interrogate whether the data can be trusted before you quote it, to follow a number back through the pipeline and the warehouse to where it could have broken, to define a metric so it means what you think it means, and to use only the statistics an analyst really needs. You’ll learn to design an experiment that can be believed, to read a p-value for what it does and does not license, to catch the bias, the confounding, and the Simpson’s paradox that flip a result, and to downgrade a correlation to the claim its design actually supports. Then you’ll learn to choose the chart that tells the truth instead of the one that flatters it, to turn a finding into a decision a busy executive will act on, and to handle user data within privacy law and without letting a biased model launder old harm into a new decision. The progression moves from reviewing a single number, to reviewing an experiment and an inference, to reviewing how the whole story reaches a decision and whether that decision is fair. There is almost no tool syntax here and a great deal about questions, trust, inference, communication, and the specific ways an agent’s analysis goes wrong. For the analyst who is going to vibe-analyze either way and wants to be the one who catches what the model missed.
Who this book is for
This book is for: analysts, aspiring analysts, and the engineers and product managers who do analysis on the side, who have realized that an AI agent will write the SQL, the Python, and even draft the chart and the summary, and that their real edge is now the judgment the agent does not have, deciding whether a number is true, whether an experiment can be believed, and whether the story it tells should change a decision, and who want that judgment without grinding through another tool tutorial.
Everything inside
- Chapter 1: How I realized my job wasn’t writing queries anymore
- Chapter 2: The question they ask me is never the one that matters
- Chapter 3: Why the query is right and the number is wrong
- Chapter 4: Where does a number break before it reaches me?
- Chapter 5: The question that cost eleven dollars
- Chapter 6: Everyone knows what an active user is, until I have to define one
- Chapter 7: Why the average lies about your users
- Chapter 8: When a result looks like proof and isn’t
- Chapter 9: Why “statistically significant” can mean nothing
- Chapter 10: How the same numbers can flip the answer
- Chapter 11: The one word that turns a number into a lie
- Chapter 12: The chart that lies with real numbers
- Chapter 13: My analysis was right and someone else got the decision
- Chapter 14: The query you’re not allowed to run
- Chapter 15: How catching one unfair model showed me the analyst I’d become
