We nowadays tend to recognize the dangers of blindly following tradition, yet in virtually all areas of mathematics we continue to adhere to notational lunacy, and pass it on to future generations starting as early as in junior high school.

This lunacy is so deeply ingrained in our minds that we see nothing wrong with the following expression:

Let’s decipher the above. We start with a number x, apply f, square the result, and finally apply sin. Yet how do we recover this order of operations from the symbols above? Starting at x in the middle we move left to f, right to the , and far left to the sin.

Although we read and write left-to-right, and so naturally imagine time evolving in this direction, the notation f(x) (due to Euler) means the opposite: start with x, then apply f. Even worse is when we have an expression like f(g(h(x))). The very first operation is written on the very right, and the final operation is written on the very left, in complete contrast to how we normally think.

Worse still is that we do read some operations left-to-right, like raising to a power, and so we end up mentally zig-zagging across mathematical writing to even decipher mathematical expressions.

But worst of all is the following: when we write a function , A denotes the input domain, and B denotes the output domain: input on the left, output on the right. As we’d expect in a sane world. If we also have a function , this left-to-right notation suggests we should be able to follow the arrows (A to B to C) to compose f and g. Yet, of course, the composite is called gf rather than fg. When reading gf we are meant to mentally reverse the direction of the arrows, so that and , although we never write this way!

This is psychotic behaviour, and we should put a stop to it. I suggest we forget tradition, and instead succumb to notational common sense to write expressions like x.f.g rather than g(f(x)).

This may seem bizarre at first, but at least one group of scientists adopted this notation years ago. In object-oriented programming languages a mental shift is required, where one stops thinking that functions have certain inputs, and instead starts to see that inputs have certain functions. In doing so, the left-to-right notation becomes completely natural, and we would write deck.sort.pop to sort a deck of cards and remove the last one, rather than pop(sort(deck)) as we’d expect in the mathematicians’ ridiculous notation.

I believe that a notational shift would make all of our lives easier in the long run, but of course it will likely never happen. Just as negative charge will always correspond to an excess of electrons, Euler’s notation is an unfortunate convention that is simply known by too many people. As a result, already-difficult subjects like category theory are made even more maddening for silly reasons, and high school students have far more difficulty learning precalculus than they should.

Help us, Barack Obama, we need Change.

(Problem 1.) You are trying to determine a secret integer x between 1 and n inclusive, using only queries of the form “is x = a (mod b)?” for integers a and b of your choosing. How many queries are required to determine x in the worst case?

And more restrictively,

(Problem 2.) Same as above but you are only allowed to make queries where b is prime.

The comments section seems to reveal at least a partial solution, but these are fun problems, so at least give them a shot before reading the solution there (or here.) If you’d like the solution spoiled anyway, read on!

First let’s talk about a problem that has a more intuitive solution. Namely, the same problem of finding an integer between 1 and n, but where we are allowed to ask questions of the form “is x ≤ a?” for our choice of integers a.

Computer scientists in the audience should recognize the solution: first, find out about the midpoint. That is, whether x ≤ n/2. Whatever the answer is, we split the search space in half! That is, we originally knew that x must be one of n values. After the first query, we know that x must be one of n/2 values: either 1 ≤ x ≤ n/2 or n/2 < x ≤ n. Either way, we can simply repeat the process again! After the second query, we know it must be one of n/4 values. And so on. For example, for 1 ≤ x ≤ 8, the following sequence of questions would reveal that the answer is x = 3.

It should be easy to convince yourself of the fact that you can only make these divisions lg(n) (that’s log-base-2*) times before you determine x.

Furthermore, this is the best we can do: any algorithm that finds x using only true/false queries requires at least lg(n) queries in the worst case. Why? Essentially, because this is how many bits there are in the binary expansion of x. Take n = 8 for example. If you could determine any 1 ≤ x ≤ 8 using only two queries in the worst case, then any such x could be written uniquely using only 2 bits, which is impossible.

But let’s go back to Problem 1. Notice that the above argument applies for any algorithm that may only make queries about x whose answers are boolean. So we may immediately say that at least lg(n) queries are required by any algorithm that solves it.

However, is the answer again exactly lg(n)? Our binary search problem had the “nice” property that no matter what the answer to our previous query was, we could always construct another query that cut the search space in half again. Any boolean query may be thought of as a filter that eliminates a certain portion of the search space (which here is simply the set {1, 2, …, n}.) But not all queries may have this nice property. For example, if we were only allowed to ask questions of the form “is x = a?” then we would need to ask n – 1 questions in the worst case—because no matter in what order we guess numbers, our first n – 1 guesses could be wrong.

It turns out that the queries in Problem 1 do indeed have this nice property. The algorithm works like so:

1. Ask whether x = 0 (mod 2). Doing so cuts the search space in half.
2. If the answer above was Yes, then ask whether x = 0 (mod 4). Since the previous answer was Yes, it must be the case that either x = 0 (mod 4) or x = 2 (mod 4), and the answer to this query again cuts the search space in half.
3. If the answer above was No, then ask whether x = 1 (mod 4). From the same reasoning as above, we again cut the search space in half.
4. Repeat the process using mod 8, 16, 32, etc., crafting the appropriate query.

It shouldn’t be too hard to find the correct question to ask in each case. As an example,

This process can always be completed in lg(n) queries, which is the answer to Problem 1.

Note, however, that we always use a modulus of the form , so this solution doesn’t help us solve Problem 2, where the modulus must always be prime! Don’t panic; I will discuss a solution to this problem in my next blog post.

* Here I’m assuming that n is a power of 2 to make the formula simpler, but if it isn’t then the correct answer is simply the ceiling of lg(n). That is, round lg(n) up. In fact, whenever I write lg(n) you can assume this is what I mean.

I think, in a sense, we are the accelerators. As we smash ideas together sometimes we discover the further structure of our fundamental particles—our complexity classes.

Duuuuude.

def encrypt_password
  ...
  self.salt = md5("#{Time.now.to_s}-#{login}")
  ...
end
 
def generate_confirm_hash
  ...
  self.confirm_hash = md5("#{Time.now.to_s}-#{email}")
  ..
end

Pro-tip for hiring managers: ask candidates to identify problems in the above code, and smack any who fail to do so.

1
2
3
4
5
6

int main ()
{
  int x = 0, y = 0;
  y = ++x + 10;
  printf("%d, %d\n", x, y);
}

That’s easy, it’s exactly what you expect: “1, 11″.

What about if you replace line 4 with the following?

y = x++ + 10;

Again, easy! This time, the expression x++ evaluates to the value of x before it is incremented, so the answer is “1, 10″.

Now, what about this?

1
2
3
4
5
6

int main ()
{
  int x = 0;
  x = ++x + 10;
  printf("%d\n", x);
}

This is no different than the first example, except that we’ve chucked the variable y, so the answer is “11″.

Last question: (and I’m sure you can see where this is going) what if you replace line 4 with the following?

x = x++ + 10;

I’m obviously trying to trick you, so you must realize that the answer is surprisingly also “11″, but can anyone tell me why it is so?

Extra credit: explain what happens if you replace line 4 with this seemingly-equivalent statement:

*(&x) = (x++) + 10;

The content of the theorem relates to what strength of pseudo-randomness is “good enough” to fool circuits of a significantly limited size and depth into thinking the pseudo-randomness is truly random. As an analogy, the D in HD video is becoming ever and ever more H (currently we’re up to a resolution of 7680×4320) but our brains have a finite capacity and our eyes aren’t perfect lenses. Increase the resolution high enough, and eventually we won’t be able to tell the difference between the HD video signal and a “true” view of the world. Similarly, we’d like to know whether certain limited models of computation can tell apart real randomness from approximate randomness.

The Linial-Nisan conjecture restricts us to logical circuits used to compute answers to decision problems/boolean functions F¹ made up of AND, OR and NOT gates², where the number of gates is allowed to be polynomial in size³  but the depth of the circuit must remain constant across all instances.

This class of circuits is called AC⁰, and these conditions are more restrictive than they first seem. You might point out that any boolean function whatsoever can be computed by a depth-2 circuit (for example, in disjunctive normal form.) And that’s true, but the number of gates then required might be superpolynomial in size. For example, even doing something as relatively simple as computing the parity of n bits cannot be done in AC⁰—you either need to increase the depth of the circuit as the problem grows in size, or have access to more complicated gates, or use more than a polynomial number of gates.

The question, then, is what kind of probability distributions over {0,1}ⁿ look identical to the uniform distribution to AC⁰ circuits. In particular, the Linial-Nisan conjecture asks about distributions that are uniform when you restrict them to any k < n bits, but are no longer uniform as a whole. For example, consider  chosen uniformly, and . The last variable is determined entirely by the others so the n bits as a whole aren’t uniformly random, but if you pick any k = n-1 bits at a time you do get a uniform distribution. Do such distributions look uniform to AC⁰ circuits? And if so, how small can k be? (Imagine a distribution half of whose bits are determined by the others. Can such a fundamentally non-random distribution be detected by an AC⁰ circuit?)

Mark Braverman proved that k can be very low indeed. It need only be polylogarithmic (for example, of order for depth-2 circuits, in m being the number of gates.) In general, the exponent is quadratic in the depth to which we restrict ourselves. Asymptotically, this is tiny! And many suspect this bound is still too high, as suffices for depth-2 circuits, leading to the suspicion that the tight bound is .

How does Braverman prove this result? He constructs low-degree polynomials that sufficiently approximate and “sandwich” the function F in question (think of the squeeze theorem from your Calculus 1 days), which was previously known to be sufficient to prove the result.

The first kind of approximating polynomials are due to Linial, Manser and Nisan way back in 1993. These LMN polynomials are created by taking the canonical, polynomial expansion of the function (for example, by interpolation) and simply chopping off the high-order terms. Such polynomials approximate F very well on average, but may not be equal to F at many, if any points. In other words, there is a small distance between them.

LMN approximation (red) to a boolean function F (black)

The second kind of approximation is due to Razborov and Smolensky. This time, the approximating polynomial agrees with the function F on most inputs, but it may behave wildly wherever it doesn’t.

Razborov-Smolensky approximation (blue) of the same function

(In fact, the jumpiness in the blue graph can be far more wild than the picture suggests, making it an awful approximation at these points.) At any rate, neither of these approximations behave quite the way we want them to. What we’d like is a low-degree approximation of F that agrees with it on most inputs (as in the blue approximation) and doesn’t stray too far away when it doesn’t (as in the red approximation.)

What Braverman discovered is that although the blue approximation can be very, very wrong at some points, this error can be checked in AC⁰! In particular, there’s a AC⁰ function ε that takes on the value 1 whenever the blue approximation disagrees with F (and possibly at some other points that give false positives, but not too many.) This function ε tells us when we need to correct the blue approximation. Then, Braverman applies the red, wavy approximation to ε and takes its negation, giving us a low-degree polynomial which (nearly) vanishes whenever the blue approximation is inaccurate.

Approximation of 1-ε (orange)

Then he just multiplies the blue and orange functions together to create a new approximation! Doing this kills the blue guy whenever he misbehaves, resulting in a polynomial that approximates F in the nice ways we wanted, and which is still of low-enough degree to be usable.

That’s pretty much all there is to it! All that’s left to do is to fill in the gaps with some algebraic magic, and the claim is proved. Not bad at all.

¹ For example, is x prime? Or, does the input graph have a vertex of degree 5?
² We allow the AND and OR gates to take any number of inputs, rather than only two.
³ For example, the circuit that computes the answer for inputs of size n could have n¹³ gates.

deck.sort_by{ rand }

…or to pick out certain elements of one…

deck.find_all{ |card| card.suit == Clubs }

…or to seamlessly cache computations.

def average_earnings
  @average_earnings ||= some_lengthy_computation
end

(Above, the ||= operator acts analogously to the familiar += operator. So if the instance variable @average_earnings already has a non-nil value, it is returned without any further computation. If on the other hand it is nil, then some_lengthy_computation is performed, @average_earnings is set to it, and returned.)

In addition, there are also a number of ridiculously short applications written in it, including a web server in 70 lines of code, a message board application in 500 lines, and its slightly more verbose successor.

In addition to these, I present a proof of concept of my own: Mathematica and Maple-like symbolic differentiation in about a hundred lines of code.

The idea is to have a Function class, which is subclassed by actual functions (Cos, Exp, Constant, etc.) as well as by operators (Addition, Multiplication, etc.)

Ruby allows us to overload operators, so we can add, subtract, multiply, and divide functions naturally, and define “f + g” as an instantiation of the Add class (which we will define later!)

Each subclass of Function must implement two methods: (1) diff, which returns another Function object (its derivative), and (2) the indexing method [], so given a Function object f and number x in its domain, we can evaluate f(x) via “f[x]“.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

class Function
  # Skeleton. Returns: another Function
  def diff
  end
 
  # Skeleton. Returns: a number
  def [](index)
  end
 
  # Begin algebraic operations
  def +(a)
    Add.new(self,a)
  end
 
  def *(a)
    a = Constant.new(a) if a.kind_of? Numeric
    Multiply.new(self,a)
  end
 
  def /(a)
    a = Constant.new(a) if a.kind_of? Numeric
    Divide.new(self,a)
  end
 
  def -(a)
    self + a * -1
  end
 
  def -@
    self * -1
  end
  # End algebraic operations
end

All binary operations are initialized the same way and have the same basic structure: they act in some way on two functions specified upon creation of the object.

1
2
3
4
5
6

class BinaryOperation < Function
  def initialize(f, g)
    @f = f
    @g = g
  end
end

Multiply subclasses it as follows:

1
2
3
4
5
6
7
8
9

class Multiply < BinaryOperation
  def diff
    @f * @g.diff + @f.diff + @g
  end
 
  def [](x)
    @f[x] * @g[x] 
  end
end

We can similarly define Add and Divide. For any operation, we simply need to know how to differentiate the operation, and how to evaluate it.

We can also define some actual functions. For example,

1
2
3
4
5
6
7
8
9

class Sin < Function
  def diff
    Cos.new
  end
 
  def [](x)
    Math.sin(x)
  end
end

1
2
3
4
5
6
7
8
9

class Cos < Function
  def diff
    -Sin.new
  end
 
  def [](x)
    Math.cos(x)
  end
end

1
2
3
4
5
6
7
8
9

class Exp < Function
  def diff
    self
  end
 
  def [](x)
    Math.exp(x)
  end
end

Once we have all of our operations and functions defined, we can start computing and plotting functions and their derivatives,

1
2
3
4
5
6
7
8
9
10
11
12

f = Cos.new / Exp.new
g = f.diff
 
(0..10).each do |a|
  x = a.to_f/10
  puts "ft#{x}t#{f[x]}"
end
 
(0..10).each do |a|
  x = a.to_f/10
  puts "f't#{x}t#{g[x]}"
end

With a little additional code, we could also print out functions symbolically.

Not bad for less than a couple hundred lines of code.