I’ve spent the last year at Echo as the Product Manager for our patient facing products. And, the more I think about it, the more I believe that traditional product development cycles need to be treated differently when working in healthcare.

Experimentation is at the heart of a normal product development cycle - and with good reason. Building product is hard, and usually, it takes at least a few ‘cycles’ to find the right fit. So when building products, not only do we want to experiment, we want to experiment quickly.

Experimenting usually (but not always) takes the form of split testing, multivariate testing, or A/B testing. In other words: some users have access to a feature, and others don’t. This gives a statistically robust answer to our questions.

This type of approach makes sense in the context of basket optimisation, engagement or even retention (Topic for another day: this definitely doesn’t mean it’s a good thing. For example, there’s definitely a lot questioning going on in social media about the types of products we’ve built chasing engagement metrics).

Running experiments is commonly accepted as a solid way to develop products by everyone from Airbnb to Skyscanner and Pinterest.

But what if the product you’re building affects patient outcomes?

For example, I’ve recently been discussing ways that our product could help people take their medication better.

All well and good right? Product Management 101 tells us that the natural next step in the product development cycle would be to work through hypotheses, then design experiments around them.

However, in this example the experiment outcome we are discussing is “people in the control group took their medication 10% better than those who did not”.

Which….really isn’t ok. It’s simply not something we would want to do.

With more of our health moving online, suddenly as Product Managers we’ve started playing God.

Do we really have the right to determine who is in or out of an experiment that impacts someone’s health? Is this really what users thought they were signing up for when they downloaded our app?

Instead of putting on the God Hat, I’d argue that as Product Managers in healthcare we should think more deeply about our product development processes.

Changing how we think about product development

As Kim Goodwin compellingly explained, the Nuremberg Code gives good direction on how to build products with the human impact - not just metrics - in mind.

Rather than jumping straight in, we should start by asking ourselves some questions:

• Is experimenting of benefit? (to the user or society)
• Is it the only way we can learn?
• Is the risk proportional?
• What kinds of harm are possible?
• How will you minimize harm?

If you’re satisfied that your answers are leading you in comfortable direction, then it’s time to start thinking about how to (in)validate your hypothesis.

There’s a couple of alternative ways that I’ve started using as alternatives to the ‘experiment on users’ approach:

1. Find clinical research that deals with similar principles

Every solid product hypothesis is based on an underlying principle (not a specific implementation).

This means that it’s pretty likely that someone has already done a version of your work for you. There is heaps of publicly available healthcare research available; Explore it, usability test it, and enhance it. Build on what’s already there with the additional insights that technology can bring.

2. Look at established metrics in healthcare

If you’ve thought of it, probably you’re not alone. I’ve recently been looking into adherence, and it turns out - unsurprisingly - that I am not the first person to think about this. There are lots of academic papers explicitly looking at measurement, and limitations of each. Even better, these are metrics that have been developed with ethical oversight that’s required in clinical research. This obviously cuts out a huge swathe of work that otherwise you’d need to deal with.

3. Look at other ways of measuring impact

Instead of defaulting to split testing, start by investigating alternative methods such as prototyping, diary studies, interviews, surveys, and ‘opt in’ split tests, ...there are so many tools out there. Explore them!

As more and more of healthcare moves online, things should (finally!) start getting easier for patients.

This also means that the ethical burden on building these types of products needs to be considered. As Product Managers, let’s just not accidentally start playing God with people’s health.

Instead - breathe - and think through how the choices we make around about process, product and patient outcomes play together in the real world.