Benchmarking Part I: Why you might want to do it and how you can collect the data

Intro

As I continue the series of articles on the projects we specialize in at SiVan Consulting, I thought I would tackle benchmarking.

This short article will cover the design and data collection process, followed by an article on the analysis, reporting, and use.

If you haven’t participated in a benchmarking exercise before, it’s probably helpful to understand “how the sausage is made” to get an idea of what is and isn’t feasible in the data-gathering phase.

But firstly…

Why conduct a benchmarking exercise

Comparing yourself to your competitors is a valuable way to assess your current situation as a life sciences company.

It could help you identify processes you could adopt, allocate resources (people, money, time) more effectively, identify and abandon/avoid harmful practices, and support the development of business cases for increased funding to your team or project.

Examples of where you might conduct a benchmarking exercise in a life sciences company include:

Product development efficiency: if you’re a med-tech company, you could compare yourself to competitors for the output of new medical devices over a series of years and look to innovators with a higher rate.
The clinical development team in a pharmaceutical company may examine competitors’ clinical trial lengths to determine whether there is something to learn from quicker ones.
As a Market Access team, if you’re struggling to get reimbursement or win tenders in certain geographies, you may look to competitors’ actions to improve your pricing effectiveness and see if other factors are involved.
Your competition’s supply chain and manufacturing capabilities could be assessed as a generic manufacturer to improve operational efficiency.

What do we benchmark?

As a first stage, you will want to decide your benchmarks—what characteristics, aspects, or values you are measuring.

You might look at how medical device companies organize their R&D teams, how clinical development teams prioritize resources, and how market access teams capture and incorporate data from clinical trials for reimbursement submissions.

During the project’s design phase, developing a longer wish list of Benchmarks to collect is normal.

However, just because you can measure a benchmark in your organization or it’s something you want to know, it doesn’t mean it will be possible to collect it externally.

On the flip side, as the data collection process is initiated, the team may identify other valuable data points that can consistently be collected.

If the information is public, quantitative benchmarks are generally more straightforward to compare, whether you’re gathering the number of new devices a MedTech company launches per year, the length of clinical trial by phase, or the time to reimbursement for new drugs per market.

If the data are not public and primary research is needed, you can sometimes get a range / rough estimate during phone interviews depending on the topic’s sensitivity and the respondent’s willingness to answer.

Qualitative benchmarks provide nuanced information you don’t get from numbers that help with the analysis and decision-making. This could include how groups are organized (by country, TA, or product), who has responsibility for decisions, or how manufacturing sites are chosen.

They help identify soft factors, such as culture and leadership styles; can provide context for quantitative factors and provide leading indicators through future plans and emerging practices.

However, even if they are publicly available, they can be more challenging to measure and compare between companies. There is a large amount of variability, and you may not be able to compare all of the companies you are benchmarking; often, it is complex to analyze with a large amount of subjectivity based on the team conducting the research.

Who do you benchmark?

The next step is to identify which competitors you will benchmark.

The first thought of many teams designing a project like this will be to pick direct competitors.

For example, if you’re a mid-size MedTech company, you may look at those making similar devices or use a financial marker like a market cap or revenue to find equivalent-sized organizations.

However, you may miss learning opportunities from smaller or bigger organizations or those working in tangential areas.

Often, valuable knowledge can be gained from companies that sell different products or are 10 times bigger or smaller. The team’s mindset has to be open to learning from different types of companies and bringing that learning back to your organization.

Ultimately, the number of competitors depends on time and budget, but ideally, 4 would be a minimum to give enough data to analyze and draw trends, with 6-8 being a better number. Then, splitting for alternate benchmarks/direct competitors, I would advise picking 2-3 alternates, with the remaining as traditional competitors.

Where, when and how do we collect the data?

Multiple techniques and sources are used to collect the information for a benchmarking exercise.

For secondary research, valuable open sources include:

Public databases, including patents
Scientific papers
Competitors websites and financial filings
Product specifications
Clinical trials websites
Conferences/tradeshows websites
Social media, including Linkedin profiles
Interviews with executives (either video or written)

Many subscription databases that store financial analyst commentary, drug profiles and other data can also be used.

Regarding primary research, surveys or phone interviews with ex- or current employees, consultants or vendors who work with multiple companies, and academics or journalists are all valuable for gathering qualitative data and ranges/estimates for quantitative data.

If timing enables it, they should be staggered near the end or after the secondary research is done. This enables the creation of hypotheses based on secondary research that can be tested during the calls.

It is always worth checking if anyone in your organization has worked recently at the competitors and can contribute through internal interviews as well. These should be conducted before external interviews and can be a great way to help shape the questions to ask.

Internal data collection

It is crucial to have the internal data available alongside the external collection. If an external team is used to conduct a project, there are two approaches for sharing this.

One is to share it upfront.

The benefit is that it gives an idea of values and types of answers you could get and provides an initial series of hypotheses (we do X this way because of Y). The potential drawback is biasing the project team’s thinking, and one of the reasons to use external support is to provide a fresh view.

The other option is to wait until the end of the external data collection, reducing the bias but potentially making the initial data collection harder, especially for “sense-checking” externally collected data.

Having used both approaches, I would say these pros and cons need to be weighed on a project-by-project basis, and there is no right or wrong answer.

Storing Data

Another important factor to consider is where and how to store data.

These projects can generate a large repository of information that can quickly become unwieldy if not planned for. But, any system needs to adapt as benchmarking projects often do not follow a linear path.

At SiVan Consulting, we typically use Notion, as its flexible series of databases are easy to interlink. It also has intelligent search features, categorising and is easily customizable to make adjustments as the project expands.

However, a well-organized SharePoint or Teams space can also work, especially with some of the free plugins available, and sensible creation of an index/contents file.

Now what?

Hopefully, this has been a helpful look behind the scenes for those who haven’t done a benchmarking exercise, or if you have, you picked up something new.

With the initial methodology and organization outlined and research initiated, it’s important to consider how to present these data for exploratory analysis within the team early on in the collection process. We’ll cover this in more detail in the next article.

As with any of the topics I post on, I’m always happy to schedule a call (Booking Link) to discuss any aspect in more detail to help you understand or shape a potential project.