You can play with the results here; for an afternoon’s coding, I’m quite pleased with it as a proof of concept even though it doesn’t always do a fantastic job of solving the problem! As implemented, each run generates a fixed obstacle set of a dozen disks (in black) and an initial population of 25 candidate solutions. The ‘genome’ is 28 bit, split 10/9/9 for x co-ordinate, y co-ordinate and radius. Disks that overlap an obstacle or go off screen are awarded a fitness of zero and painted red; otherwise the fitness function is simply the radius. In line with the tutorial, I pick pairs by roulette selection based on this fitness (so red disks shouldn’t survive into the next generation), breed by crossover at bit 20, and mutate about 1 in 200 bits.

However, for this particular set up the crossover approach doesn’t seem ideal- offspring will inherit almost exactly the co-ordinates of one parent and the radius of the other. This often means large areas of the screen are missed unless a chance mutation moves a disk, which then has to be lucky enough to produce a few offspring in the next generation. It might work better if I generated offspring bit by bit (say, taking bits from each parent a certain percentage of the time), rather than the simple cut currently in use. Fortunately, it shouldn’t be hard to experiment in this way; ideally, I’d build several algorithms into the applet and offer a choice at runtime, as well as control over the assorted variables. If nothing else, it’s teaching me a lot about processing, which I continue to be impressed with.

The topics covered are: Diffie-Hellman and one-way functions for key exchange; the generic Discrete Logarithm Problem and BSGS algorithm; scalar multiplication- addition chains, fast exponentiation, m-ary methods and windowing; group law implementations, Side-channel attacks and the Edwards form.

I’ve discussed several of these ideas elsewhere on this blog, as well as the cryptanalysis ideas I mentioned on the day but which are not in the notes. I also refered to a recent real-world example of a side-channel attack; see this story from The Register for details.

The Euclidean Algorithm

Euclid’s algorithm recovers the greatest common divisor of two numbers a and b- that is, the largest x with the property that x|a and x|b (we write
x = gcd(a, b)). Thus, anything that is true of divisors of a and b in general is true of their g.c.d in particular.

To this end, recall that if x|a and x|b then x|a ± b. Further, if x|b then x|λb for any integer λ. Combining the two, we have that

x|a and x|b ⇒ x|(a ± λb) for all λ     (1)

How does this help us? Since gcd(a, b) = gcd(b, a), we can assume w.l.o.g that a ≥ b. Then by the quotient-remainder theorem, we can write a as a = qb + r for some integers q, r with 0 ≤ r < b. Rearranging gives us r = a -qb, so by (1) we have

x|a and x|b ⇒ x|(a – qb) = r     (2)

So in particular

gcd(a, b)|a and gcd(a, b)|b ⇒ gcd(a, b)|r     (3)

So, to solve the question of finding gcd(a, b), we can instead solve the question of finding gcd(b, r). Notice that b ≤ a and r < b, so this second question is easier.

Repeating this process, the pairs of terms keep decreasing until eventually we have the question “find the g.c.d. of r and 0″ for some r. But as anything divides 0, gcd(r, 0) = r. So there is always a final question; that question is easy to answer; and its answer is the same as the original g.c.d. we were looking for!

Worked Example

Suppose then that we want to find the g.c.d. of 51 and 36. We first appeal to the quotient-remainder theorem:

51 = 36 + 15

Thus any divisor of 51 and 36 (including the greatest) is also a divisor of 15. So we turn our attention to the easier problem of the g.c.d. of 36 and 15. Again by quotient-remainder

36 = 2 * 15 + 6

so it suffices to find gcd(15,6). Noting that

15 = 2 * 6 + 3

we discover that this is the same as the g.c.d. of 6 and 3. But

6 = 2 * 3 + 0

so that’s simply gcd(3,0)=3. Thus we have

gcd(51, 36) = gcd(36, 15) = gcd(15, 6) = gcd(6, 3) = gcd(3, 0) = 3

The Extended Euclidean Algorithm

Typically, one tabulates their progress through the algorithm more compactly; using the previous example we have

51 = 36 + 15      (4)

36 = 2 * 15 + 6   (5)

15 = 2 * 6 + 3     (6)

6   = 2 * 3 + 0     (7)

With such a representation, we can now tackle a related problem – given a, b, finding x, y such that

ax + by = gcd(a, b)

For our example, this means finding x, y with the property that 51x+36y = 3.

Notice that we do not have an identity in terms of 3, 36 and 51 anywhere in the table. But from (6), we do have an identity for 3 in terms of 15 and 6:

3 = 15 – 2 * 6

Then (5) tells us that

6 = 36 – 2 * 15

so we can rewrite our result for 3 in terms of 36 and 15 instead of 15 and 6, and so on.

In this way, we can chain upwards through the table to an identity for 3 ultimately in terms of 51 and 36- this process is known as the extended euclidean algorithm. Whereas the Euclidean algorithm works down from a and b, through simpler and simpler terms to the g.c.d., so the extended version works up from that g.c.d. through increasingly complicated terms to an expression in terms of a and b.

For our example, this works as follows:

3 = 15 – 2 * 6                               by (6)

   = 15 – 2 * (36 – 2 * 15)            by (5)

   = 5 * 15 – 2 * 36           (simplifying)

   = 5 * (51 – 36) – 2 * 36            by (4)

   = 5 * 51 – 7 * 36           (simplifying)

Linear Diophantine Equations

Why is all this useful? In number theory, the study of Diophantine equations concerns finding integer solutions to equations, where possible. Typically we seek to classify such equations based on whether we can find infinitely many such solutions, only finitely many, or none at all. The general question as to whether solutions exist is actually undecidable, but in various special cases we can say something.

Armed with the extended euclidean algorithm, we can tackle the case of linear diophantine equations in two variables:

For fixed integers a, b, c, do there exist integer solutions x, y of ax + by = c ?

Existence

Suppose c = λgcd(a, b) for some integer λ . Then, by the extended Euclidean algorithm we have x’, y’ with the property ax’+by’=gcd(a, b). So x = λx’, y = λy’ solve the linear diophantine equation, since

ax + by = aλx’ + bλy’ = λ(ax’ + by’) = λgcd(a, b) = c

Conversely, suppose c is not a multiple of gcd(a, b). Then, if the equation had solutions x, y then we’d have that gcd(a, b)|a|ax and gcd(a, b)|b|by so gcd(a, b)|ax + by = c, which is a contradiction.

Hence, the linear diophantine equation in two variables ax + by = c has integer solutions if and only if c is a multiple of gcd(a, b).

Number of Solutions

Assuming solutions exist, how many are there? Although the g.c.d. is unique, suitable pairs (x, y) with ax+by = gcd(a, b) (and hence solutions to the linear diophantine equation) are not. To see this, consider one of the standard tricks of analysis, adding and subtracting the same thing:

ax+by = λgcd(a, b) ) ax+by+ab-ab = λgcd(a, b) ) a(x+b)+b(y-a) = λgcd(a, b)

That is, if (x, y) satisfy the equation, so do (x + b, y – a). Which means we can apply the result again, giving (x+2b, y-2a). These will always be pairs of integer solutions, and so the existence of just one pair (x, y) gives us an infinite family (x + ib, y – ia) of solutions.

Hence we conclude that the linear diophantine equation in two variables ax+by = c has infinitely many integer solutions if c is a multiple of gcd(a, b), and none otherwise.

The primary school answer is to do just that: given g, compute 2g=g+g; then compute 3g=2g+g and so on, for n steps. But this rapidly becomes tedious, and isn’t (I hope) how we’d perform such a calculation in our heads: instead, we’d try to take bigger jumps. For instance, to compute the number 16g, we can find 2g, add that to itself to reach 4g, and again for 8g, hitting 16g after just 4 steps. Had we wanted 17, we’d then just add g again.

The sequence of intermediate values correspond to an addition chain for n: this is a sequence starting at a₀=1 and ending at a_l=n with the property that for any term a_k there exist terms a_i,a_j with a_k=a_i+a_j. That is, every term is the sum of two earlier terms (not necessarily distinct), so we can reach ng by computing each a_kg in turn without needing anything fancier than our addition law.

As a side effect, an addition chain also tells us how to exponentiate if we know how to multiply, since x^a+b=x^ax^b so we can build up to xⁿ by finding the powers x^a_k in turn.

Thus our tedious sequence for 16 is 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16, with the more sophisticated one being 1,2,4,8,16. If I asked you to compute 16x for a number x, you’d probably do something different: compute 10x (easy in base 10!) and 6x, then add those. With access to a multiplication procedure, this is definitely more efficient: but without that luxury it can still be used to give an addition chain for 16, by finding chains for 10 and 6. These are 1,2,4,5,10 and 1,2,3,6 respectively, so we can merge them into the chain 1,2,3,4,5,6,10,16.

We describe the length of the chain as l: that is, there are l terms after the 1. Notice that at worst, therefore, we need n-1 terms. If we move as quickly as possible by taking a_k+1=a_k+a_k=2a_k then the greatest value we can reach after m terms is 2^m, as demonstrated with the chain for 2⁴=16. However, if we can compute n and m in l_n,l_m terms, it needn’t take as many as l_n+l_m+1 terms to compute n+m, since there may be common terms in the chains. Finally, as defined an addition chain needn’t make use of all it’s terms: for example, 1,2,4,6,8,16 is still a chain for 16, although the 6 isn’t needed. We will obviously be interested in chains without any such wasted terms.

The observation for powers of two motivates a first attempt at an efficient algorithm for building addition chains. if n is 1, then we have nothing to do: the chain 1 suffices. Otherwise, we can work recursively: for an even number n=2k, we ask for the chain for k and then add 2k to the end; whereas for an odd number n=2k+1, we ask for the chain for k and add 2k then 2k+1 to the end. This will require at most log₂n calls, and each step adds 1 or 2 entries to the chain.

This can also be captured by considering the binary representation of n, which will also indicate whether we hit an even or odd number at each stage. Suppose n is of the form

where each d_i is either 0 or 1, and d_k=1 so there are no leading zeros. Then we start the chain with 1, and read from left to right, ignoring the leading 1: if we see a 0, then we double the last term of the chain; if we see a 1, add 1 to this new term as well. More formally:

Input: the binary expansion of n.

Output: an addition chain for n.

Set a₀=1
Set t=1
for i from k-1 down to 0:
    Set a_t=2a_t-1 and t=t+1.
    If d_i=1:
        Set a_t=a_t-1+1 and t=t+1
return a₀…a_t

For instance, with n a power of 2 we will see only a run of zeros, and thus perform the appropriate number of doublings, for a chain of length log₂n. For n one less than a power of two, the expansion consists entirely of 1s, costing 2log₂n. For an ‘average’ number where each digit has around a 50% chance of being a 1 or a 0, the chain will therefore be of approximate length (3/2)log₂n. More formally, we can define the Hamming Weight v(n) to be the number of 1s in the binary expansion of n; then the chain will have length floor(log₂n) + v(n) -1.

Is this optimal? A counterexample suffices to show it is not, the first being 15, which is 1111 in binary. Using the binary expansion we get the sequence 1,2,3,6,7,14,15 of length 6 (=3+4-1 as claimed). But the sequence would be 1,2,3,5,10,15 of length 5 and hence 1 shorter.

Nonetheless, our binary chain gives a much better upper bound than the naive chain of (n-1) steps, and is very simple to implement both in logic and memory. The idea also generalises to arbitrary bases, as follows.

Suppose n is of the form

where each d_i is in the range 0…m-1, d_k non-zero. Then we can compute an m-ary addition chain for n:

Input: the m-ary expansion of n.

Output: an addition chain for n.

Set a=0
Start the chain with 1,2,3,…,m-1.
for i from k down to 0:
    Extend the chain to m*a (if not already present) and set a=m*a.
    Add a+d_i to the chain (if not already present), and set a=a+d_i.
return chain.

Notice that a tracks the last term computed, and that extending the chain to m*a is itself an addition chain problem: this makes some 2^k a good choice for m, since this extension can then be accomplished in k additions.

The shorter chain for 15 arises from its ternary expansion: considered as (120)₃ 1*9+2*3+0*1, we have an initial chain of 1,2 then apply the loop. With a=0,d₂=1 we set a=3a=0 then a=a+1=1; neither of these need to be added to the chain; for a=1, d₁=2 we set a=3a=3 which requires the addition of 3 to the chain; then a=a+2=5 which extends the chain to 1,2,3,5; finally for a=5,d₂=0 we set a=3a=15, adding first 10 then 15 to the chain; then a=a+0=15 so we terminate with chain 1,2,3,5,10,15.

The optimal choice of m depends on n, since although there will be less steps (log_mn) for greater m, each step will require more additions in the computation of m*a. Nor are m-ary expansions the end of the story- there are various other tricks available to finesse addition chains. However, there is no known algorithm for finding shortest addition chains- and a generalisation of the problem to finding shortest addition sequences (chains that contain a desired list of values n₁,n₂,…,n_k) is NP-complete. Searching for short chains is therefore an interesting challenge, and is constrained by the fact that unless you are precomputing them for future use, any shorter chain must offer a speedup in computation greater than the time taken to find it.

There are, however, bounds for the length of the chain:

Where v(n) is the Hamming weight as before.

References/Further Reading

Knuth- The Art of Computer Programming Vol. 2: Seminumerical Algorithms.

Bos and Coster- Addition Chain Heuristics in Advances in Cryptology- Crypto’89 (Lecture Notes in Computer Science) (Available via SpringerLink.)

Gordon- A Survey of Fast Exponentiation Methods. (Available via citeseer.)

Discrete Logarithm Problem (DLP)
Let G be a cyclic group, with operation ⊕. Let [n] represent (n-1)-fold application of ⊕ i.e., [2]g=g⊕g, [3]g=g⊕g⊕g etc.

Given g,h from G, the discrete logarithm problem is to find t such that [t]g=h.

The baby-step, giant step (BSGS) algorithm is a generic algorithm for solving the DLP- that is to say, it makes no appeal to properties of the group involved, merely calculating abstractly with ⊕. There is a theoretical upper bound to the effectiveness of generic algorithms, and BSGS approaches are of that order of magnitude.

The simplest demonstration of BSGS (the original by Shanks) assumes that g generates G, with both of known order n: recall that the group order is simply the number of elements it contains, whereas the order of an element g is the least n such that [n]g=id_G, should such a value exist. The order n of an element always divides the order of the group N, with equality when g generates G. Between the two there is also the exponent, the least value e such that [e]g=id_G for all g in G; for cyclic groups this is N, but for products of cyclic groups (the structure of groups of rational points) it is the lowest common multiple of their respective orders, and thus, although a divisor of N, may be significantly smaller than it.

So, if we can find an order m rational point on a curve, we know that the cardinality of the group of rational points is zero modulo m. By testing random points and taking the lowest common multiple of their order we can usually find the exponent e of the group. With luck, but not always, this will be large enough that when combined with bounds on the cardinality the latter is established exactly (as with modular information in the SEA algorithm).

But it is no good attempting to do so using an algorithm that requires the order of the point! There are BSGS algorithms for the DLP which can handle unknown order, for a given g we can apply these to solve for h=id_G in the cyclic group generated by g. Provided the algorithm gives the minimal t such that [t]g=h=id_G in G=<g>, then t is the order of g.

One such algorithm, by Terr, is given in Cohen and Frey’s Handbook of Elliptic and Hyperelliptic curve cryptography (my current bible!) It relies on the following observation:

Let T_n be the nth triangular number: that is, defined by the recursion T₁=0, T_n+1=T_n+n.

Then any non-negative integer t, there are unique integers j and k with t=T_j+1-k with 0≤k<j.

To see how, notice that there is a unique j such that t lies in the interval (T_j , … , T_j+1]=(T_j+1-j , … , T_j+1-0] and that this interval has width j so there is an appropriate choice of k.

We then suppose that t satisfies [t]g=h. Then we have [T_j+1]g=h⊕[k]g. So, instead of testing each [t] in turn until we hit equality, for a given j we need only test the ‘giant step’ [T_j+1]g against the set of ‘baby steps’ β={β_i}={h⊕[0]g,…,h⊕[i]g,…,h⊕[j-1]g. If that j yields no matches, we move to j’=j+1: by keeping track of [j]g at each iteration, the new T_j’+1 and the extra h⊕[j'-1]g are found by a single group operation each. Hence this is attractive from both storage and time complexity perspectives: we need only record the baby steps β, [j]g and a single giant step at any given iteration j, whilst the time complexity is of order t^1/2.

Terr’s BSGS variant for the DLP
Finds t such that [t]g=h for g generating G of unknown order, h from G.

Initialise β={β₀=h}, γ=δ=id_G, j=0. ((For each iteration, we have γ=[T_j+1]g and δ=[j]g)
Then loop over j as follows:

• Increment j by 1, then update:
• Set δ=δ⊕g=[j-1]g⊕g=[j]g.
• Set γ=γ⊕δ=[T_j]g+[j]g=[T_j+1]g.
• If j≥2 then
  • For s from 0 to j, if γ=β_s then return T_j+1-s
• Add β_j=β_j-1⊕g to β

I’ve implemented this generic DLP algorithm and an order-finding version (which requires you to catch g=id_G) as Maple procedures for arbitrary groups- details on that will be in the next post! BSGS-based approaches become impractical at finite field sizes well within the grasp of SEA for counting on elliptic curves; but are of interest in the higher genus case which lacks an Elkies procedure.

Fortunately there is an excellent book – Blake, Seroussi and Smart’s Elliptic Curves in Cryptography which covers this topic thoroughly. As usual, attempting to implement the procedures myself has uncovered (and repaired) many gaps in my comprehension of the algorithms involved. I’m finally getting correct answers, albeit for small examples, so I’ve written up some summary notes on the book and a worked example in Maple. It still requires a high degree of interaction to get to the final result; my understanding is that SAGE (on linux) implements the SEA algorithm, so it doesn’t seem worth trying to tie it all together into a standalone Maple procedure- this would be a programming challenge, but wouldn’t add anything to my mathematical understanding!

Notes: [SEA.dvi]
Appendix (Maple procedures): [SEA_tools.mpl]
Example (Maple worksheet): [SEA.mw]