One mystery of drug productivity at large pharma is the persistent low productivity despite the fact that there are tremendously talented scientists at every large pharma company. Their expertise is often encyclopedic, and their creativity is often very evident. Despite this, productivity at large companies have been less than impressive. The productivity appears to be low in comparison to small biotechs, but even more shocking, the net return on investment seems to be negative. Studies by multiple entities, such as Ernst and Young, have demonstrated that the return on investment is lower than the cost of capital for the large companies.
One possible explanation is that higher productivity at small biotechs is illusory. There are some studies showing that on average small companies are no more productive than large pharma companies. This may be true – it may simply be that the variability is higher, that there are both very productive and very unproductive small companies, simply because smaller number of employees means that it is easier to be an outlier (See my post “Small Sample Paradox”).
But it probably goes deeper than that. One driver for this is likely to be the decision-making process at large companies.
Consensus Decision-Making
At most large companies, the decisions are made by consensus. Every person on the team, and there can be twenty or more people on teams, have to agree before the team can move forward. And for each of them to agree, their functions (their bosses) have to agree. This means that the decisions default to the lowest common denominator, namely the least risky decision. In addition, because everyone is responsible for the decision, no one was.
The reason why consensus decision making doesn’t work is that in an innovation based industry where 90% – 95% of drugs fail, you must take calculated risks to be successful. There is an apocryphal story about a senior executive at Pfizer who would vote against every project because he knew that if he did so, he would be right 95% of the time (because 95% of drugs fail), while if he voted for a project even 10% of the time, he would be wrong 50% of the time.
If you rely on consensus decisions, you usually end up with the decision that the most conservative person on the team is comfortable with. That is a problem. I can’t tell you where on the spectrum of risk any given decision should sit, but one thing I can tell you is that the least risky decision is almost never the right one (except when it comes to safety). In a risky industry such as drug development, the only way to have no failures is to have no successes.
It’s like appendectomies. When I was a medical student, I was taught that a good surgeon will, in appendicitis cases, take out healthy appendixes at least 20% of the time. In other words, a good surgeon has such high index of suspicion that he will be wrong 20% of the time and cut into a patient who doesn’t have appendicitis. If you’re not wrong at least 20% of the time, you’re not being aggressive enough and some your patients will die from appendicitis.
In addition, in a consensus decision making system, what often happens is that the functions horse trade, so that the decision that is the least painful to every function, and pain of the decision is relatively equally distributed. The problem is that often, the right decision is one that is beneficial to one function at the cost of another one. A claim for the drug, for example, may be worth so much that the clinical group should conduct a difficult study. Or stability may be so fickle for a drug from a CMC standpoint that marketing needs to take a hit on the shelf life.
Left to consensus decision making, the decision often ends up such that it’s a compromise between two functions. Once again, I can’t tell you where on the spectrum the right decision is, but what I can tell you is that it’s rarely at the halfway point between what two functions want.
Of course, once a decision is made, all the functions need to line up behind the decision. In a healthy corporate culture, this happens. In an unhealthy one, there are a lot of foot dragging and passive aggressive behavior. This leads to consensus decision making solely to insure that foot dragging is avoided.
And it gets worse.
Not only do you need horizontal consensus across team members, but then you have to obtain vertical consensus. What I mean by that is that you have to go up through several layers of approval up the chain of command. And once again, I can’t tell you what the right decision is, but if six people up the chain of command all agree with the decision, it is almost never the right decision because once again, only the least risky decision will pass muster.
Genentech during the 2000’s had a run of successes. Its success rate in drug development was about 80%, which is more than an order of magnitude higher than the standard 5%-10% seen in industry.
One of the keys to Genentech’s success, in my opinion, was its decision making process.
Unlike many other companies, which make decisions by consensus, Genentech always had one final decision maker. One of its strengths was that decisions were never consensus driven. Usually, the team leader had that responsibility, but in some cases, the portfolio committee’s head, such as Sue Hellman or Myrtle Potter, had the final say. In some cases, the committee vote would be 20 to 1, but if the 1 was the committee head, that single vote carried the day. The team leader had the authority and the responsibility to make the final decision. The decision maker could delegate the decision, and he/she was responsible for making sure everyone had a chance to present their case. Of course, the decision maker had the responsibility of listening to and weighing all arguments and data, but the final decision was his/hers.
Of course, once a decision is made, all the functions need to line up behind the decision. In a healthy corporate culture, this happens. In an unhealthy one, there are a lot of foot dragging and passive aggressive behavior. This leads to consensus decision making solely to insure that foot dragging is avoided.
The second component of decision making that was done well at Genentech was that decisions were based on data. Opinion carried some, but not much, weight. The decisions were scientific and fact-based. In some cases, this might have been carried too far, since at a certain point, it becomes impossible to further de-risk projects with additional data (see my comment about complicated vs. complex decisions), but overall, the data-based decisions turned out to be high quality decisions.
The third component of decision making that Genentech excelled at was cutting losses. They didn’t fool themselves by performing secondary ad hoc analysis to data dredge for a positive signal. If the primary endpoint was missed, they moved on to the next molecule. Of course, this was easier to do for them than some other companies because they had 20 promising molecules waiting for every 1 that was dropped, but they avoided the futile efforts that many other companies exerted, chasing a phantom positive signal from post hoc analyses.
I’ve tried to incorporate that decision-making philosophy wherever I go. In addition, I tell the teams that I don’t have to agree with their decisions. I only have to understand it and be sure that it is thoughtful. The way I see it is, if we’ve hired the right team, and provided the right corporate context for the decision, then the likelihood that the team that spends every day thinking about the program would be right is much higher than the likelihood that I, spending a few hours a month thinking about the program, would be right. Of course, that means that senior management has to be very transparent and provide the corporate information the teams need to make the decisions.
The other important aspect of decision-making, which I learned from my mentor, Hal, is to distinguish between good/bad decisions vs. right/wrong decisions. A decision, if made thoughtfully and with all the appropriate input, is a good (high quality) decision. The decision may turn out to be wrong–for example the drug may not work in the end–but it remains a good decision. Some companies punish good decisions that turn out to be wrong. That fosters risk aversion and can paralyze the organization. With a 5% success rate across industry, most good decisions will turn out to be wrong decisions.
What we want to avoid is the urban legend sometimes related in pharmaceutical circles about the Senior VP at a large pharma who voted against every project because he knew he would be right 95% of the time if he did that. He knew that if he voted for a project even 10% of the time, he would be wrong 50% of the time. That, of course, is the exactly wrong attitude in drug development. You can’t run a pharmaceutical company with the goal of not failing, you have to run it with the goal of succeeding.
Data-Driven Decisions
Another thing Genentech did right was to have data-driven decision-making. They did not make decisions based on whose voice was the loudest, or who was the most persuasive. They required that team bring data to the decision-making meetings. This seems similar to what many successful companies do. P&G, where I worked at the beginning of my career was extremely good at this, and many successful internet companies seem to do this. In fact, apparently, if you say, “you seem to feel really strongly about that” at some of these internet companies, it’s an insult, implying that you don’t have data to back up your proposal.
Data-driven decision not only tend to drive toward the best decision, but instills discipline–the team has to do the work in accumulating and digesting data instead of relying on opinions.
Data-driven decision also drive something that’s even more important than making right decisions: it drives reversing bad decisions. This is critical in drug development because 95% of decisions turn out to be wrong.
There is a persistent sunk cost fallacy issue in drug development. It’s not easy to walk away after 10 years and $200 million dollar investment. The natural urge is to “save” the drug if at all possible. So people do post-hoc analysis, looking for a glimmer of hope. And often, there will be some subgroup that responded very well to the drug. And people will say, “sure, we did a lot of data mining, but look, the survival was doubled in that subgroup of men between 43 and 56 years of age who have diabetes and live north of where they work. And the p-value is 0.0001.” Needless to say, those heroic efforts at second tries almost never work, and end up costing another $200 million.
Data-driven decision-making forces companies to leave bad drugs behind. The CEO of Genentech was a fearless leader in this. I remember when entire departments were dismantled after decades of effort when it became clear that the direction of research was not going to pan out. Few companies have the intestinal fortitude to do that.
It is very easy and exciting to start projects. And very painful and difficult to stop them. The ability to change on a dime once the data speaks is one of the most success criteria. When I worked at OneWorld Health and had an opportunity to observe how Bill Gates worked. He is one of the most persistent people you’ll ever meet but also data-driven. He hates to be wrong, but when the data speaks, he will instant change his plans 180 degrees. Art Levinson, the CEO of Genentech and one of the giants in biotech history was like that. And the story about Andy Grove getting out Intel out of the memory chip business is well known.
Fearon, in his book Dead Companies Walking, talks about how athletes often make poor CEOs. Athletes become great because they are persistent. Apparently, according to Fearon, they do not quit even when the odds clearly turn against them and the best course of action would be to adopt a different strategy. Curt Schilling is certainly an example of that, persisting with his game company until he lost all of his money–$50 million, believing until the end that the game was going to be successful.
I abhor making decisions based on opinions. I tell people who work for me, “If you and I have two opinions on a decision, then I am going to go with my opinion, not because it is necessarily better than yours but because I have no reason to believe that it is any worse. If you want to change my mind, bring data, not opinions.” I find that if they can bring me data, then it is often an excellent sign that they are right and I am wrong.
Limits of Data-Driven Decisions
Having said all that, there are limits to data-driven decisions. We need to distinguish between simple, complicated, and complex decisions. (See here for detailed explanation)
Simple decisions are problems like “what shall I order for dinner?”
Complicated decisions are decisions like, “how can we sen a rocket to the moon?” These types of decisions have an answer. It might take a lot of data and analysis, but eventually, you can solve the equation.
Complex decisions are problems like “how can we win this election?” There is no one answer. There can be multiple answers, and it is difficult to predict how your actions will influence the results. And when you make a move, your opponent makes a move. In those cases, further data or analysis won’t help. You have to just try a pilot and then see how it works. Most internet companies approach their businesses in this fashion, as described in the book, Lean Startup.
And other decisions are both complicated and complex.
Scientists tend to see all decisions as complicated decisions, and are sometimes caught in analysis paralysis. When the course of action is not clear, they will continue to accumulate more data instead of trying something and seeing the results.
Levels of Decision-Making Delegation
The last point I want to make about decision-making is that delegation of decision-making is critical. One of the most important tasks a manager has is to delegate and manage other people. It is related to Situational Leadership model.
When I was at a company that was undergoing a turnaround, I saw a Wartime CEO in action. Some of the employees praised him as an outstanding listener, interested in hearing what people had to say. Others complained that he never listened and that he ignored them.
What was happening was the following. There are multiple levels of decision delegation, and various writers have categorized them into several different schema, but basically, it boils down to the following.
Level 1 : I Decide
The manager decides on the course of action, and tells the team what to do. If the manager is skilled, he may try to sell the decision to the team, either by persuasion or by whipping up enthusiasm like the people who come up through sales often do. Typically, the task will be outlined in detail, with the expectation that the employee not deviate from the directive.
Level 2: You tell me the facts and I decide
The manager solicits input from the employee about the problem. How much market share have lost? How late is the delivery and why? Then manager then decides.
Level 3: You give me a recommendation and I decide
Rather than asking for data and solving the problem, the manger asks the employee to assess the situation, solve the problem, and to present a recommendation.
Level 4: We agree on the decision
The manager and the employee will discuss and come to a consensus on the decision. For this type of decision, it is critical that the manager not overrule the employee if the agreement is that it will be a consensus decision.
Level 5: I will recommend, but you decide and let me know
The employee is authorized to make the decision. However, the manage might offer his advice, and he wants to be informed of the decision.
Level 6: You decide
The manager is not even informed of the decision. The decision is fully delegated.
It is critical to be clear on what level of delegation you are authorizing. Sometimes the employee will assume that he has the authority to operate at a higher level than intended. It is also important to let the employee know that if they need advice at Level 6 that the manger is still available to advise.
In the company I mention above, what was happening was that the CEO was soliciting a tremendous amount of factual input about the problem, but he had no interest in hearing recommendation about what the decision should be. This was because the people giving advice were often the same people who got the company into the trouble in the first place. He was interested in understanding the problem, but he formulated the solution himself, because that was what he was best at.