TL;DR: Competitive analysis is an effective tool for making decisions about performance optimizations.
Main audience: managers, engineers, researchers, and anyone with a competitive mindset.
Read time: 7 minutes.
Decisions about performance optimization are very common in a competitive environment. In a business context, managers face such decisions in processes such as acquisition (should I get this high-throughput machine?), planning (how far out should I plan this effort?), and recruiting (where should I set the bar for recruiting?), to name a few. Engineers and office workers face them in their choices of tooling for everyday tasks. Consumers face them when considering whether to buy a lower-end or higher-end product. And researchers face them when considering whether to try a low-hanging-fruit or a far-out direction.
By their nature, performance optimization decisions are critical to long-term success. Thus, the question of how to make them is an important one in a competitive environment. This post deals with this question.
A Simple Case
Let’s start with a simple case to introduce the concepts we will need. We will later see how to generalize and apply the concepts to many other practical cases.
We consider the case known as the Ski rental problem. In this problem, each day the skier needs to decide whether to rent skis that day for a price of $1 or to buy skis for $10 and never have to rent again; but on any given time the skier does not know how many more days they will end up skiing before quitting. The worse-case situation is that the last day of skiing ends up being the same day the skier decided to buy. It turns out the best (deterministic) strategy is to buy on day 10, which guarantees the skier pays 90% more than necessary, namely $19 instead of $10 had the skier bought on day 1. While there are better randomized strategies, we do not need to consider them here. The important points to note are:
- This is a case of making a decision about performance optimization.
- The correct decision is arrived at using a competitive analysis.
Let’s see what competitive analysis means.
Competitive analysis
A competitive analysis is a comparison of the performance of two competitors operating in exactly the same potential situations, however one has a superpower of knowing the future and one does not. Let’s call them the imaginary and realistic competitors, or Ivan and Raphael by nicknames. Clearly, Ivan’s performance is no worse than Raphael’s. The competitive analysis focuses on the situation(s) that lead to the greatest performance gap between Ivan and Raphael. Naturally, Raphael wants to minimize this gap, i.e. to minimize regret due to not knowing the future.
Let’s use the language of competitive analysis for the above simple case. Raphael’s strategy is to buy on day 10. A potential situation is characterized by the day after which the skier quits, i.e. the first situation is quitting after day 1 etc. The future N that will materialize is an unknown for Raphael. The situation of quitting on day N results in Raphael’s performance being $(10+N-1) cost. What would Ivan, knowing the future N, do to optimize performance? If N <= 10 then Ivan would rent for N days then quit, for a cost of $N, while if N > 10 then Ivan will buy on day 1, for a cost of $10. We want to compare Raphael’s and Ivan’s performance, R and I, using the competitive ratio, defined by C = R / I. If N <= 10 this ratio is 1 while if N > 10 it is (10+N-1)/10. The worst-case situation that maximizes C is quitting on day 10, with C = 1.9, i.e. 90% extra cost.
Generalizing the Analysis
There are many possible ways to generalize the competitive analysis. Here are some:
- Instead of a competitive ratio of costs, we may have a case that calls for a competitive ratio of gains.
- Instead of focusing on a worst-case situation, we could assign a probability to each potential situation and consider the competitive ratio of the expected performances given these probabilities.
- Instead of considering the competitive ratio, we may prefer to consider a competitive difference D = R – I or some other competitive measure.
- Instead of analyzing just one exact case, we may analyze a parameterized family of cases to understand how the competitive ratio changes with the parameters of the family.
The appropriate generalizations to use depend on the particular case to analyze.
Applications
Finally, we can go back to the practical cases presented at the top and view them with a competitive analysis in mind. The following table lists example performance factors and unknowns for each competitive question.
| Competitive question | Performance factors | Unknowns (which Ivan knows!) |
|---|---|---|
| Should I get this high-throughput machine? | Cost (or gain) gap between the options of getting and not getting the machine. | Time until failure and work load of the machine. |
| How far out should I plan this effort? | Costs and benefits of planning vs agility. | Availability of resources, and the cost and benefits of committed resources. |
| Where should I set the bar for recruiting? | Cost of passing on a good candidate vs that of accepting a bad one. | Availability of good candidates in the market and cost of failing to recruit. |
| Which tools should I use for my recurring tasks? | Costs of setting up a given tool and of repeated using of the tool vs the quality of its production. | Frequency of recurrence of future tasks and their deadlines. |
| Which product should I consume? | Price vs value of a given product. | Frequency of use and maintenance of the product. |
| Should I research a low-hanging-fruit or a far-out direction? | Effort of and potential achievement from researching a given direction. | Strength and timing of success signals, and of intermediate rewards. |
Conclusions
We looked into decisions about performance optimization and introduced competitive analysis as an effective tool for making such decisions. A competitive analysis of a simple case demonstrated the main concepts:
- The imaginary and realistic competitors, Ivan and Raphael.
- The potential situations, which the competitors may face.
- The unknowns for Raphael that are known to Ivan.
- The competitive ratio, and similar competitive measures.
We demonstrated these concepts are applicable to many practical cases that involve decisions about performance optimization.