The 37% Rule

Candidates arrive one at a time in random order. You can rank each against those already seen, you must say yes or no on the spot, and nobody can be called back. Choose how many to reject outright; the rule then takes the first candidate who beats all of them.

Chance of the best, rejecting k P(k)=knβˆ‘i=k+1n1iβˆ’1P(k)=\frac{k}{n}\sum_{i=k+1}^{n}\frac{1}{i-1}
Best cutoff, large n kβˆ—β‰ˆnek^*\approx\frac{n}{e}
Its chance P(kβˆ—)β†’1eβ‰ˆ0.3679P(k^*)\to\frac{1}{e}\approx0.3679
Loading interactive simulation...

Lesson

The theory β€” The 37% Rule

In the secretary problem, n candidates arrive in random order, each is compared only with those already seen, and each must be accepted or rejected on the spot, for good. The cutoff rule rejects the first k and then accepts the first candidate better than all of them.

Where the formula comes from

  1. Split by where the best is. Write B for the position of the best candidate and W for ending with it. B is equally likely to be any of 1 to n.
    P(B=i)=1n,i=1,…,nP(B=i)=\frac{1}{n},\qquad i=1,\dots,n
  2. A best among the first k is lost. The rule never looks back, so those positions contribute nothing.
    B≀k:P(W∣B)=0B\le k:\quad P(W\mid B)=0
  3. A best at position i > k is reached only if nobody between k + 1 and i βˆ’ 1 was taken. That happens exactly when the best of the first i βˆ’ 1 sits among the first k, since only then does nobody in between beat the benchmark. Any of the i βˆ’ 1 is equally likely to be that one.
    B=i>k:P(W∣B=i)=kiβˆ’1B=i>k:\quad P(W\mid B=i)=\frac{k}{i-1}
  4. Add over the positions.
    P(k)=βˆ‘i=k+1n1nβ‹…kiβˆ’1=knβˆ‘i=k+1n1iβˆ’1P(k)=\sum_{i=k+1}^{n}\frac{1}{n}\cdot\frac{k}{i-1}=\frac{k}{n}\sum_{i=k+1}^{n}\frac{1}{i-1}
  5. Let n grow. With x = k/n the sum becomes an integral and the chance becomes βˆ’x ln x. Its derivative, βˆ’ln x βˆ’ 1, is zero at x = 1/e, where the chance is 1/e β‰ˆ 36.79%.
    Pβ‰ˆx∫x1dtt=βˆ’xln⁑x,ddx(βˆ’xln⁑x)=βˆ’ln⁑xβˆ’1=0β€…β€ŠβŸΊβ€…β€Šx=1eP\approx x\int_{x}^{1}\frac{dt}{t}=-x\ln x,\qquad\frac{d}{dx}\left(-x\ln x\right)=-\ln x-1=0\iff x=\frac{1}{e}
Assumes
Only ranks are seen, never scores; the order is uniformly random; a rejected candidate cannot be recalled and an accepted one always says yes; and anything short of the very best counts as a loss.

Problems solved in full

  1. Three candidates, every order listed 3 steps

    Three candidates arrive in random order. Write each order as the ranks in arrival order (1 = best). Rejecting the first and then taking the first who beats everyone before, in how many of the six orders do you end with the best? Check the answer with the formula.

    Open this problem in the tool

    1. The six orders:

      123,132,213,231,312,321123,\quad 132,\quad 213,\quad 231,\quad 312,\quad 321
    2. The first candidate sets the benchmark. In 213 and 231 the best beats the 2; in 312 the 1 arrives second and is taken. In 123 and 132 nobody beats the 1 that was rejected, so the last is taken; in 321 the 2 beats the 3 and is taken before the 1 arrives:

      213β†’1,231β†’1,312β†’1;123β†’3,132β†’2,321β†’2213\to1,\quad 231\to1,\quad 312\to1;\qquad 123\to3,\quad 132\to2,\quad 321\to2
    3. The formula with n = 3 and k = 1:

      P(1)=13(11+12)=13β‹…32=12P(1)=\frac{1}{3}\left(\frac{1}{1}+\frac{1}{2}\right)=\frac{1}{3}\cdot\frac{3}{2}=\frac{1}{2}

    Answer

    3 of the 6 orders, a chance of 1/2, against 1/3 for taking the first candidate.

    P(1)=36=12>13=P(0)P(1)=\frac{3}{6}=\frac{1}{2}>\frac{1}{3}=P(0)
  2. Four candidates, every cutoff 2 steps

    With four candidates, find the chance of ending with the best for each cutoff k = 0, 1, 2, 3, and the best cutoff.

    Open this problem in the tool

    Show solution
    1. k = 0 takes the first candidate, who is the best with chance 1/4. For k β‰₯ 1 use the formula:

      P(1)=14(1+12+13)=1124,P(2)=24(12+13)=512,P(3)=34β‹…13=14P(1)=\frac{1}{4}\left(1+\frac{1}{2}+\frac{1}{3}\right)=\frac{11}{24},\quad P(2)=\frac{2}{4}\left(\frac{1}{2}+\frac{1}{3}\right)=\frac{5}{12},\quad P(3)=\frac{3}{4}\cdot\frac{1}{3}=\frac{1}{4}
    2. Compare the four:

      P(0)=624,P(1)=1124,P(2)=1024,P(3)=624P(0)=\frac{6}{24},\quad P(1)=\frac{11}{24},\quad P(2)=\frac{10}{24},\quad P(3)=\frac{6}{24}

    Answer

    Reject one, then take the first who beats it: 11/24 β‰ˆ 45.83%.

    kβˆ—=1,P(1)=1124β‰ˆ0.4583k^*=1,\qquad P(1)=\frac{11}{24}\approx0.4583
  3. Kepler's eleven 3 steps

    In 1611 Johannes Kepler set about choosing a second wife and, over nearly two years, considered eleven candidates. With the cutoff rule and n = 11, which cutoff is best, what is its chance, and which candidate is the first the rule can take?

    Open this problem in the tool

    Show solution
    1. Evaluate the formula around n/e = 11/e β‰ˆ 4.05:

      P(4)=411(14+15+β‹―+110)=411Γ—1.0956=0.3984P(4)=\frac{4}{11}\left(\frac{1}{4}+\frac{1}{5}+\dots+\frac{1}{10}\right)=\frac{4}{11}\times1.0956=0.3984
    2. k = 4 is the largest; k = 3 and k = 5 are both lower:

      P(3)=311(13+β‹―+110)=0.3897,P(5)=511(15+β‹―+110)=0.3844P(3)=\frac{3}{11}\left(\frac{1}{3}+\dots+\frac{1}{10}\right)=0.3897,\qquad P(5)=\frac{5}{11}\left(\frac{1}{5}+\dots+\frac{1}{10}\right)=0.3844
    3. With four rejected, the fifth candidate is the first the rule can accept. Kepler did marry the fifth, but not by this rule: Ferguson's account of his letter has him persuaded to propose to the fourth after meeting the fifth, being turned down, and going back. The rule forbids going back.

    Answer

    Reject four, chance 39.84%; the fifth is the first candidate the rule can take.

    kβˆ—=4,P(4)β‰ˆ0.3984k^*=4,\qquad P(4)\approx0.3984
  4. Many candidates: where 1/e comes from 3 steps

    For large n, write the share rejected as x = k/n and find the best x and the chance it gives. Then check against n = 100.

    Open this problem in the tool

    Show solution
    1. The sum is a Riemann sum for 1/t on [x, 1]:

      knβˆ‘i=k+1n1iβˆ’1=xβˆ‘j=knβˆ’11nβ‹…1j/nβ‰ˆx∫x1dtt=βˆ’xln⁑x\frac{k}{n}\sum_{i=k+1}^{n}\frac{1}{i-1}=x\sum_{j=k}^{n-1}\frac{1}{n}\cdot\frac{1}{j/n}\approx x\int_{x}^{1}\frac{dt}{t}=-x\ln x
    2. Set the derivative to zero:

      ddx(βˆ’xln⁑x)=βˆ’ln⁑xβˆ’1=0Β β‡’Β x=1e,βˆ’1eln⁑1e=1eβ‰ˆ0.3679\frac{d}{dx}\left(-x\ln x\right)=-\ln x-1=0\ \Rightarrow\ x=\frac{1}{e},\qquad -\frac{1}{e}\ln\frac{1}{e}=\frac{1}{e}\approx0.3679
    3. For n = 100, n/e β‰ˆ 36.79, and the exact best cutoff is 37:

      P(36)=0.37101,P(37)=0.37104,P(38)=0.37080P(36)=0.37101,\qquad P(37)=0.37104,\qquad P(38)=0.37080

    Answer

    Reject a share 1/e β‰ˆ 36.79%; the chance tends to 1/e. At n = 100 the best cutoff is 37, with chance 37.1%.

    xβˆ—=1e,Pβ†’1eβ‰ˆ0.3679;n=100:Β kβˆ—=37,Β P(37)β‰ˆ0.3710x^*=\frac{1}{e},\qquad P\to\frac{1}{e}\approx0.3679;\qquad n=100:\ k^*=37,\ P(37)\approx0.3710
  5. Left with the last candidate 3 steps

    A hundred candidates, the first 37 rejected. Find the chance that the best is among the rejected, the chance that the search reaches the last candidate, and the chance that the last candidate is then the best.

    Open this problem in the tool

    Show solution
    1. The best is equally likely to be anywhere, so it is among the first 37 with chance:

      kn=37100=0.37\frac{k}{n}=\frac{37}{100}=0.37
    2. The search reaches candidate 100 exactly when nobody from 38 to 99 is taken, that is when the best of the first 99 is among the first 37:

      knβˆ’1=3799β‰ˆ0.3737\frac{k}{n-1}=\frac{37}{99}\approx0.3737
    3. The difference is the case where the search reaches the end and the last candidate is the best:

      3799βˆ’37100=379900β‰ˆ0.0037\frac{37}{99}-\frac{37}{100}=\frac{37}{9900}\approx0.0037

    Answer

    37%, 37.37% and 0.37%: more than one search in three ends with the last candidate, and almost never because the last is the best.

    0.37,0.3737,0.00370.37,\qquad 0.3737,\qquad 0.0037
  6. The lowest expected rank, three candidates 3 steps

    Three candidates, and the aim is now the lowest expected rank (1 = best), not the best itself. Find the optimal rule by backward induction and its expected rank.

    Open this problem in the tool

    Show solution
    1. A candidate seen i-th whose rank among those seen is j has expected overall rank j(n + 1)/(i + 1). The third candidate must be taken, and has expected rank (n + 1)/2:

      j(n+1)i+1=4ji+1,c2=3+12=2\frac{j(n+1)}{i+1}=\frac{4j}{i+1},\qquad c_2=\frac{3+1}{2}=2
    2. At the second candidate, take it if its expected rank is no more than 2, otherwise go on:

      i=2:Β Β j=1:Β 43<2,j=2:Β 83>2;c1=12(43+2)=53i=2:\ \ j=1:\ \tfrac{4}{3}<2,\quad j=2:\ \tfrac{8}{3}>2;\qquad c_1=\frac{1}{2}\left(\frac{4}{3}+2\right)=\frac{5}{3}
    3. At the first, taking it gives 2, more than 5/3, so go on:

      i=1:Β Β 42=2>53Β β‡’Β c0=c1=53i=1:\ \ \frac{4}{2}=2>\frac{5}{3}\ \Rightarrow\ c_0=c_1=\frac{5}{3}

    Answer

    Pass the first, take the second if it beats the first, otherwise take the third: expected rank 5/3 β‰ˆ 1.67. With three candidates this is exactly the cutoff rule with k = 1.

    c0=53β‰ˆ1.6667c_0=\frac{5}{3}\approx1.6667
  7. Two candidates, scores seen 2 steps

    Two candidates have scores drawn independently and uniformly from 0 to 1, and you see each score as it arrives. Find the rule that most often ends with the higher score, and its chance. Compare it with ranks alone.

    Open this problem in the tool

    Show solution
    1. If you pass the first, whose score is x, you must take the second, which wins when it is higher: chance 1 βˆ’ x. If you take the first, it wins when the second is lower: chance x.

      S(x)=x,C(x)=1βˆ’x;S(x)β‰₯C(x)β€…β€ŠβŸΊβ€…β€Šxβ‰₯12S(x)=x,\qquad C(x)=1-x;\qquad S(x)\ge C(x)\iff x\ge\tfrac{1}{2}
    2. So take the first when x β‰₯ 1/2. Average over x:

      ∫01/2(1βˆ’x) dx+∫1/21x dx=38+38=34\int_{0}^{1/2}(1-x)\,dx+\int_{1/2}^{1}x\,dx=\frac{3}{8}+\frac{3}{8}=\frac{3}{4}

    Answer

    Take the first when its score is at least 1/2: chance 3/4. With ranks alone, two candidates give 1/2 whatever you do.

    P=34>12P=\frac{3}{4}>\frac{1}{2}

Supplementary problems, answers only

  1. With five candidates, find the best cutoff and its chance.

    Show answer

    Reject two: 13/30 β‰ˆ 43.33%.

    kβˆ—=2,P(2)=25(12+13+14)=1330β‰ˆ0.4333k^*=2,\qquad P(2)=\frac{2}{5}\left(\frac{1}{2}+\frac{1}{3}+\frac{1}{4}\right)=\frac{13}{30}\approx0.4333
  2. With two candidates and ranks only, show that every rule ends with the best half the time.

    Show answer

    Taking the first wins with 1/2; passing it means taking the second, also 1/2.

    P(0)=12,P(1)=12β‹…11=12P(0)=\frac{1}{2},\qquad P(1)=\frac{1}{2}\cdot\frac{1}{1}=\frac{1}{2}
  3. With ten candidates at the best cutoff, find the chance the best is among the rejected and the chance the search reaches the last candidate.

    Show answer

    k = 3: 30% and 3/9 β‰ˆ 33.33%.

    kβˆ—=3:310=0.3,39β‰ˆ0.3333k^*=3:\quad\frac{3}{10}=0.3,\qquad\frac{3}{9}\approx0.3333
  4. In the large-n limit, find the chance of the best when a fifth of the candidates are rejected, and when three fifths are.

    Show answer

    βˆ’0.2 ln 0.2 β‰ˆ 32.19% and βˆ’0.6 ln 0.6 β‰ˆ 30.65%.

    βˆ’0.2ln⁑0.2β‰ˆ0.3219,βˆ’0.6ln⁑0.6β‰ˆ0.3065-0.2\ln0.2\approx0.3219,\qquad-0.6\ln0.6\approx0.3065
  5. With four candidates, find the lowest expected rank any rule can reach.

    Show answer

    15/8 = 1.875.

    c0=158=1.875c_0=\frac{15}{8}=1.875

Reject a quarter or reject a half: the same chance

The curve is flat at the top, and that is what makes the rule usable. Press A quarter rejected: 35.04%. Press Half rejected: 34.91%. The best cutoff for a hundred, 37, gives 37.1%, so rejecting anywhere from a quarter to a half costs you at most 2.2 points. As n grows the curve becomes βˆ’x ln x, with x the share rejected, and at a quarter and at a half it takes exactly the same value, because ΒΌ ln 4 = Β½ ln 2 β‰ˆ 34.66%. So you need not know n well: guess it a third too high or a third too low, reject 37% of your guess, and for large n you still keep at least 34.5%.

Aim only for the best, and more than a third of the time you get the last

The rule bets everything on the very best. When the best is among the candidates you rejected, which happens 37% of the time with a hundred, nobody later can beat the benchmark, and the search runs to the end and takes whoever comes last. Press A hundred, 37 rejected: the expected rank of the candidate taken is 20.01. In 1964 Chow, Moriguti, Robbins and Samuels asked for the rule with the lowest expected rank instead. It takes a candidate who is merely near the top of those seen so far, with a standard that loosens as the line runs out, and its expected rank at n = 100 is 3.60. Theirs stays below 4 for every n this page offers and tends to 3.8695; the 37% rule's keeps growing with n, to 185.54 at a thousand.

See the scores, and the chance rises to 58%

The 37% rule uses only ranks: who beat whom. Suppose instead that you see each candidate's actual score and know how scores are spread, say uniformly between 0 and 1. Gilbert and Mosteller solved this version in 1966, and the look-without-choosing phase disappears. From the first candidate on, you take anyone who is the best so far and scores above a threshold, and the threshold falls as the line runs out: 0.5 with one candidate still to come, 0.6899 with two, 0.9240 with ten. The chance of ending with the best is 60.6% for eleven candidates and 58.29% for a hundred, and it tends to 58.02%, against 36.79% from ranks alone. Knowing what a good score looks like is worth about 21 points.

Learning path

Choosing on what you have seen

References (4)