Read case studies for their conditions, not their results

Agency pitches, vendor decks and conference talks bring success stories with them. Take a hypothetical but familiar set: a brand that doubled its pipeline with a new channel, a competitor that cut its acquisition costs with a different creative approach, a peer company that rebuilt its sales motion and grew within a year. When you are deciding where next year's budget goes, these stories are persuasive, and they should be taken seriously: most of what works in business has worked somewhere before.

A result alone does not tell you whether a tactic will work for your business. You also need to understand when the work happened, what the client brought to it, who did it and what else could explain it. Copy the tactic without those conditions and you may not get the result.

I learned this as the person telling a success story. This post covers the headline I had to correct, four conditions I would look for behind any result, and the questions to ask before you fund a playbook that worked for someone else.

DisclosureJake Jorgovan is a partner in Fieldtrip, a group of marketing agencies, which this essay links to.

The headline I had to correct

In January 2018 I published Zero to $33k MRR in 6 months, about Lead Cookie, the LinkedIn outreach service I had started pitching in July 2017. By January 15, 2018 it was at $33,168 in monthly recurring revenue, and the essay listed the lessons I credited for that.

About 18 months later I wrote a correction. The essay had valuable lessons, I said, but the headline did not hold up. Lead Cookie may have hit those numbers in six months, I wrote, but "that is misleading". None of it would have happened without the 15 years of work that came before. That included a video production agency I built with a partner in college, nine months on a healthcare startup and freelance consulting from 2015 to mid-2017, in which I slowly narrowed from a generalist to outbound marketing campaigns.

In February 2020 I said it most plainly, in a short post in The Daily, the series of one-lesson posts I wrote from 2018 to 2020. It was called "Timing counts": "The truth is, we hit a market trend at the right time." I had noticed the trend of using LinkedIn as an outreach channel right at the start of its rise, and Lead Cookie's growth was largely due to more and more marketing money going into LinkedIn outreach as a whole.

Starting the same company in 2020 would have been far harder. The market had matured, there were many more competitors, and we would not have had the case studies we had by then. I did a lot to make Lead Cookie succeed, but I also got the timing right. "When you look at any success," I wrote, "consider that this is part of the formula."

Image

Zoom

Jake’s historical account, summarized from this article. These results describe his experience and are not a performance forecast.

Four conditions to check behind a success story

The 2018 essay had useful lessons about client fit, testing and delivery. Its headline still credited six months for a result that also rested on the years before them and on the market's timing. A vendor's case study, an industry benchmark or a peer's playbook can have the same gap. Four conditions are worth asking about before you copy any of them.

When it happened

A channel that works for the first companies in it can look very different once everyone else arrives. In an October 2019 episode of my podcast, I described the competitor research I had done when I started Lead Cookie. I found maybe seven people in the space, only one of them legitimate, and people asked what LinkedIn lead generation even was. Almost two and a half years later there were too many to count. In a June 2019 daily post I wrote about the other LinkedIn lead generation companies that had asked me, as they were starting up, for the secrets. Part of it was timing, and "timing, market conditions, and industry trends are fluid and make it impossible to replicate."

So the first question about any case study is when the work ran, and what the channel looks like now. A result from the early days of a channel describes conditions your competitors have since found too. Why marketing tactics decay goes into how that happens.

What the client brought

In the 2018 essay I wrote down an equation for how to think about Lead Cookie's results, and said it applied to any marketing service: "(Your value proposition) X (Lead Cookie's Service) = Results". A client whose tagline was "I help blue collar companies automate manual processes" had a sharply defined niche, and his results were amazing. A generalist software development agency used the same service and "saw crickets". The value proposition acted as a multiplier. At that time we turned away about four in five of the leads who came to us, because their value propositions were not strong enough for the service to work.

When a vendor shows you a case study, you are seeing one client's offer multiplied by the vendor's work. Ask what that client brought: how clear its offer was, how well known its brand was, how much budget and existing demand it had. Then ask how the vendor decides which clients to take on, and where the same approach has not worked. A vendor that explains where its approach failed gives you a clearer view of the clients it can serve well.

Who did the work, and what they had done before

The 2019 correction made the same point about founders in general: "No one comes out of the gate swinging. And even if it looks like they do, they have honed their skills somewhere else that has enabled them to achieve results so quickly." My examples were other founders. Dan Norris had failed over and over again before he partnered with Alex McClafferty, who had spent years developing his skills in corporate. Russ Perry had spent years trying to run a traditional design agency before Design Pickle.

The same applies to a vendor's result. It came from particular people, with particular experience, on that account. Ask who did the work and whether the same people would work on yours. For a peer company's playbook, ask what the leader who ran it had done before, and whether your team has that experience.

Image

Zoom

The founder-age distributions summarized by NBER illustrate why a success story needs context about the person behind it. The study uses U.S. administrative records and distinguishes all founders from those of the fastest-growing firms.nber.org (opens in a new tab)

What else could explain the result

An August 2019 daily post on hindsight bias started from a fair point: "most success does come from following patterns that have been proven to work before and applying them in new ways." The danger comes after: "People have some success and then they try to teach others. But they fail to realize that it's more than what they teach that led to their success. It could be their unique situations and a bit of luck that led to success."

A case study is an explanation written after the result, by the people who got it, and a vendor's case study is also a sales document. Neither makes it wrong. The risk is that the explanation gives the method more credit than the circumstances justify. Ask what else changed for the client in the same period: a new product, a price change, a competitor leaving, a market that was growing anyway.

Model what works, then test it in your conditions

None of this is a reason to ignore other people's results. In the same June 2019 post I wrote: "Yes. Model after others. But don't get lost in comparisons to someone else who may have just nailed the timing and faced different market conditions than you."

Lead Cookie itself started that way. Before launching it, I researched the people who were writing and speaking about LinkedIn outreach, read what they published, and then tested their tactics on my own consulting practice. Within two weeks I was booking quite a few calls, and I could see the approach was simple and repeatable enough to train others to run. Only then did I put up a landing page and start offering it to people in my network.

For a marketing leader, the equivalent is to treat a vendor's case study as a hypothesis to test in your own market:

1. Agree the baseline, the time limit and the measure you will judge it by before it starts.
2. Run a small, defined test.
3. Decide whether to scale on your own result.

Read an industry benchmark the same way before you hold your team to it.

Image

Zoom

The approach described in this article, shown as a visual guide.

Which result in your current plan is borrowed from someone else's story, and do you know the conditions it came from?

((The three posts from The Daily quoted here, 163 (June 13, 2019), 193 (August 1, 2019) and 317 (February 21, 2020), are no longer online; the quotations are from the site's archive of September 24, 2026.))