1. 43
  1. 13

    Rule of five: Take five samples. You can be 93% confident the median is between your five samples.

    1. 2

      Assuming your data is normally distributed? Best make that explicit if so.

      1. 15

        No, the rule applies to 5 random samples from any distribution.

        Douglas W. Hubbard, How to Measure Anything (3rd ed.):

        There is a 93.75% chance that the median of a population is between the smallest and largest values in any random sample of five from that population.

        It might seem impossible to be 93.75% certain about anything based on a random sample of just five, but it works. To understand why this method works, it is important to note that the Rule of Five estimates only the median of a population. Remember, the median is the point where half the population is above it and half is below it. If we randomly picked five values that were all above the median or all below it, then the median would be outside our range. But what is the chance of that, really?

        The chance of randomly picking a value above the median is, by definition, 50%—the same as a coin flip resulting in “heads.” The chance of randomly selecting five values that happen to be all above the median is like flipping a coin and getting heads five times in a row. The chance of getting heads five times in a row in a random coin flip is 1 in 32, or 3.125%; the same is true with getting five tails in a row. The chance of not getting all heads or all tails is then 100% − 3.125% × 2, or 93.75%. Therefore, the chance of at least one out of a sample of five being above the median and at least one being below is 93.75% (round it down to 93% or even 90% if you want to be conservative).

        1. 5

          Cool! But just keep in mind, if we don’t know the underlying distribution, the mean (and mode) could be far from the median value. A normal distribution is the special case where median, median, and mode coincide. Log-normal data are pretty common too, but often people don’t have as good an intuition for these distributions since they aren’t as popularized. And then there are multi-modal distributions…

          Still, a good trick to know. Thanks!

          1. 3

            Thanks. That book is where I learned it from as well.

            It also has the Single Sample Majority Rule: Pick one sample. Probability is 75% that you picked it from the majority.

            I don’t find that rule that useful though. At least, I haven’t found any practical use for it yet. The book describes an academic urn experiment and suggests betting in bars. First, 75% is somewhat low to make decisions on (but the 93% above is good). Second,“majority” might be anything between 50%+1 and 100%.

            The general point of that part of the book is that even very small samples can reduce uncertainty significantly. That is a good lesson. For example, no matter how big the codebase, you could randomly pick one or five source files and apply the two rules to learn something about the project if you know nothing yet.

      2. 5

        One I often use is assuming a day has 100,000 seconds as that’s close enough for estimation purposes.

        1. 3

          Shouldn’t one use Approximate With Powers with the Rule of 72, and just make it the Rule of 70?

          1. 3

            For those who want to learn more about estimations, MIT had published ‘The Art of Insight in Science and Engineering: Mastering Complexity’ (available freely online), which is all about skill of approximation.

            1. 3

              A note on being careful about overapplying LIttle’s law: note that it’s formulated in terms of averages, and that’s what it works on. For Markovian queues (which subscription-based systems tend to look like, in the short term), the average contains quite a bit of information about the full distribution, and there are known, convenient ways to estimate the variances of the quantities involved.

              Public APIs, websites, and generally any open system will likely have traffic behaviour that is far too bursty/fat-tailed to make any meaningful inferences out of the averages of anything. If you want to handle peak traffic, you have to look at how the peaks relate to each other. There is no useable shortcut involving averages.

              1. 3

                First time I’ve heard of the rule of 72, looks pretty useful.

                1. 6

                  https://en.wikipedia.org/wiki/Rule_of_72 explains, basically it’s actually the rule of e*100 (i.e. ~69.3) but 72 has more divisors so is easier to work with.

                  1. 3

                    basically it’s actually the rule of e*100 (i.e. ~69.3)

                    I think it’s ln(2) ~ 69.

                    1. 1

                      Right you are, I was getting my logarithms mixed up.

                2. 3

                  Hey nice tricks you’ve got there! I appreciate the concrete examples. The units in the first equation should be “bps” though, and the Little’s Law section seems to have mixups with variable names.

                  1. 1

                    Whoops, thanks for the correction! :)

                    1. 2

                      Thanks for putting this together. The rule of 72 is going to come in handy.

                      While we’re doing suggestions - the queueing example with actual numbers plugged in swaps lambda and W. While still mathematically correct it did caused me to stutter for a sec reading the example since it looked like a super slow rate of entry and an insanely huge processing time.

                  2. 2

                    It’s hard not to reach for pen, paper & a calculator (M-x calc) right off the bat. Before you realize it you’re looking up exact numbers and seeing what the calculation should be.

                    I’ve been trying to force myself to do the mental math & estimation on the spo. I don’t write much/anything down and see how far off I was some later time. I’m starting to learn more of these tricks. The dumbest one I’ve done is just memorize all kinds of useful numbers & terms. Anki reminds me to practice all my SI units from time to time.

                    It’s especially helpful in conversations where people are bouncing back and forth a few different options, but everyone is acting like all options are equal. They are rarely equal.

                    1. 1

                      I find myself reaching for the calculator more, but more to check magnitude while I think. I’d like to rely on recall a bit more, when in good practice it makes for more fluid thought.

                      Two qualities of fluid BoE calculation is that it doesn’t pull you out of the zone when you are trying to be creative. Reifying the creative thought in the “will it work specifically” mindset converts the mind from ideation to editorialization.

                      The second one related to the first is that BoE calculations mostly serve as indicators that the idea won’t not work (double negative is necessary). This prunes the solution space while staying creative.

                    2. 2

                      One trick I like: when estimating area or volume, estimate the edge lengths and multiply. Humans are really bad at estimating areas and volumes.

                      1. 1

                        Or if you have a similar shape of different size where you do know the area approximately (e.g. because it lines up better with a grid or whatnot), eyeball the ratio of two comparable sides in both shapes and the area is bigger by the square of the ratio of lengths.