Modulo Errors

The winter issue of iSquared magazine is now available, with the cover article ‘The limits of Computation’ being by, well, me! I discuss Turing Machines, undecidability, intractability, and (briefly) Artificial Intelligence.

I’ve not received my own copy so can’t comment on the other articles yet, although I’ve attended a talk by interviewee Keith Briggs before which was very good, so I expect the interview will be interesting too. So I hope you’ll pick up a copy- it’s a venture well worth supporting!

Yesterday I gave another postgraduate colloquium, on the basics of public key cryptography. The slideshow I used is here [pdf], although obviously it’s missing the narrative to go with it. The first part, on sharing secret colours, was lifted from the talk described here, although I discussed the discrete logarithm problem in group-theoretic terms this time.

Mathematical study is often thought of as ‘purer’ than scientific research- instead of labs full of chemicals, fruit flies or lasers, our work could in theory proceed with nothing more than a chalkboard; rather than believing theories through weight of evidence and an absence of counterexamples, we prove theorems as undeniable consequences of our base assumptions.

But theorems rarely spring into our minds fully formed, simply awaiting proof. Instead mathematical research can often mimic the scientific method- experimenting with ideas, following promising leads, looking for tests that would break or support hunches, until they look convincing enough to attempt a proof.

This has very much been the nature of my work over the past few months- and although the actual proofs have mostly been worked out by pen and paper, with supporting calculations on a MacBook laptop, the exploratory phase requires something rather more powerful. One of the perks of being a University of Edinburgh researcher is access to the ECDF – a cluster of over 1400 processors, as illustrated above. Provided a problem can be split up into many independent sub-problems, such a resource can offer staggering reductions in computation time, by farming them out to multiple machines and running them in parallel.

For instance, I usually wish to generate very, very long lists of matrices and test them for a certain property, keeping only the much smaller subset of those with the property. A typical calculation of this type might take five days on my little Mac, assuming it doesn’t simply run out of memory in the process. But by splitting it into ten smaller calculations (each exploring a subset of the matrices), I can have the answer from the grid in 12 hours- or, splitting to 20 jobs, I can load it up overnight, and collect the answer the next morning just in time for a supervisor meeting!

Of course, a twenty-fold reduction in calculation time only hints at the power of the cluster; another 70 users could also get such a job done that night. Running all-out on a single task, ECDF could complete four years worth of calculation in a day!

Not all researchers have access to systems like Eddie, but there is one network almost everyone has access to- the internet. The idea of harnessing the power of idle home computers is nothing new: over ten years ago, I was contributing spare cycles of my (then powerful) Pentium 200 to a distributed.net project that succeeded in demonstrating the insecurity of 56-bit encryption keys, which at the time was the maximum allowed by the US government in software exports. Other famous distributed efforts include the alien-seeking SETI and the biological research project folding@home.

The potential for highly parallelised computing has surged as processor counts have grown and always-on internet connectivity becomes increasingly ubiquitous. Modern PCs often have multiple processor cores (and many homes have multiple PCs), and there has been work on exploiting highly powerful (yet specialised) graphics card technology for general purpose computation too: you can even contribute to folding@home with a PlayStation3!

However, except for a few long-range ‘big ticket’ schemes such as the ones I’ve mentioned, setting up, managing and publicising your own distributed computing project might be more daunting than solving your original problem! Fortunately, there is an increasing range of ways to connect researchers to resources. At ANTS (in connection with another cryptographic challenge) I learnt of BOINC, a general platform that has grown out of SETI: you can submit a project to the volunteer community, or manage a grid within your own organisation. All modern Apple computers include the Xgrid feature, allowing again for local grid computing with your own machines or harnessing donated cycles through the openmacgrid project. Recently, Fedora Nightlife was announced, to create a community grid from machines running the popular linux distribution.

There are limitations, of course. Not all calculations fall in to the ‘embarassingly parallel’ category that lends itself so well to distributed computing. There are also concerns about the other costs of such schemes – home PCs are not necessarily cut out for 24/7 running, and those that are will probably add a bit to their owner’s power bill. However, Bryan’s blog makes a good case for the environmental merits of distributed computing over data centres, assuming that the calculations are worthwhile and thus going to be done somewhere anyway – a claim that is perhaps easier to defend for cancer studies than the search for alien signals. Or you can think of it as a very direct form of charity in support of science- donating some electricity, processing power and wear-and-tear rather than cash. So, if like me your home features more computers than people, why not devote some of their time to volunteer computing?

On Friday I gave a geometry club seminar, speaking about some of the computational aspects of discrete-logarithm cryptography in general and as implemented for elliptic curves. My notes supplement rather than completely describe the talk, being heavier on the formalities and lighter on the narrative.

The topics covered are: Diffie-Hellman and one-way functions for key exchange; the generic Discrete Logarithm Problem and BSGS algorithm; scalar multiplication- addition chains, fast exponentiation, m-ary methods and windowing; group law implementations, Side-channel attacks and the Edwards form.

I’ve discussed several of these ideas elsewhere on this blog, as well as the cryptanalysis ideas I mentioned on the day but which are not in the notes. I also refered to a recent real-world example of a side-channel attack; see this story from The Register for details.

This entry first appeared as a writeup for Everything2.

Greedy pig is a simple maths game for groups that serves as an introduction to probability. I used it recently as a warm-up activity for a maths hour with local primary school children (around 11 years old), where it was well-received. For older students, it could provide the starting point for a discussion of topics such as the gambler’s fallacy or for a statistical investigation.

How to Play

A pair of dice are thrown, and their total recorded as a starting score for all participants. Play then proceeds in rounds. Before each round, players decide whether to stick with their current score, or continue playing. To play a round, roll a die; each player who is still in adds that many points to their score- unless a two is thrown, in which case they lose all their points. Play proceeds until all participants have decided to keep their score, or a two eliminates all remaining players. The winners of the game are the players with the highest score; it’s worth playing around three games and taking a combined total.

Practical issues when running the game

Keeping track of who’s in or out is most easily done by having students stand up if they wish to gamble or sit down if they wish to stick with their score. Apart from making it easy to spot when a player is trying to sneak back into the game, this is also good as it gives the students an idea of how confident their peers are to continue, and you’ll get lots of them wavering up and down as they try to decide!

Recording scores of players as they drop out is harder, but vital- children may try to cheat, or accuse each other of doing so, when it comes to declaring their final total. It’s definitely worth keeping a running tally of throws and totals on the blackboard -with the students doing the adding up! For smaller groups, you might be able to give out tokens or numbered cards as players save their score, but with larger groups (we had around 30 students per session) this would probably slow things down a lot. Perhaps give each student a piece of paper and a pen to write their score on (nice and large!) to hold up once they’ve sat down.

Some children are very risk adverse, and sit down almost immediately; others just stay standing until they get knocked out by a two. To make sure this isn’t due to misunderstanding the game, it’s worth doing a practice run first. It’s interesting to watch how strategies adapt as the players get more experience- particularly if the two is thrown surprisingly early or late in a game (we hit a total in the 70s for one session, which skewed things somewhat!)

Strategy

Can we say anything mathematically about when a player should sit down? That is, should we gamble a given total or not? You might want to think about this yourself before reading on.

To model this game, we can consider the expectation of a round- that is, the average outcome in the long run. Suppose then we have a total of N. Obviously, it’s only worth playing if our expected increase in the total offsets the risk of losing it all. One sixth of the time, a one will be thrown in the next round, leading to a gain of 1; with equal probability we might gain 3,4,5, or 6. So five sixths of the time, we gain some amount. But the remaining sixth of the time, we’ll hit a two and lose everything; this can be thought of as a ‘gain’ of minus N. So our expected gain by staying in is:

(1/6)*1+(1/6)*(-N)+(1/6)*3+(1/6)*4+(1/6)*5+(1/6)*6 = (1/6)*(1+3+4+5+6-N) = (19-N)/6.

Hence, for a play to be worthwhile, we need (19-N)/6 (and thus 19-N) to be positive. That is, we should be willing to gamble on a total of 19 or less, but a total of 20 or above should be banked.

Game Theory

However, as is always the case with expectation theory, this analysis depends on playing a large number of games and considering the total (or average) score across them. Playing just a few games tends to encourage an ‘all or nothing’ approach wherein players are more interested in winning in absolute terms (that is, being best in class) than the score attained in the progress.

Of course, the ideal time to bank is just before the two is thrown, thus leaving you with the maximum possible score (anyone who sat out earlier has less; anyone who stayed in scores zero). The problem is that by banking a score in a given round with the hope of winning that particular game, you are effectively gambling on it being the next throw being a two, and you’ll only be right one sixth of the time. The remaining five sixths of the time, you’ll be wrong and the others get a higher score- in a group situation, any of them could now retire with a better score than you, and in a 1-on-1 duel your opponent can bank immediately to guarantee the win.

But then, if everyone adopts this brinksmanship strategy of always staying in, then eventually they will all go over the brink and score zero. Depending on how the payoffs are modelled (which is of course crucial!) this two-player version of a round of Greedy Pig can be interpreted in game-theoretic terms as follows. If neither player has an advantage over the other, either by ending the game with a tied score, or by proceeding to another round, then assign them a score of 1, unless both players tied with zero, in which case score 0. Else, if one player wins the game this round and the other loses, the winner scores 2 and the loser 0. Mixing in the probabilities of winning or losing depending on a play of stick or gamble, we get a payoff bimatrix:

Notice that for player 1 the ‘gamble’ row dominates the ’stick’ row (and equivalently for player 2 in columns), and thus each player must gamble despite the fact that they each prefer the outcome of both sticking (score 1) to both gambling (score 5/6). Thinking of sticking as cooperating, and gambling as defecting, this is precisely the famous prisoner’s dilemma!

Variations

More advanced versions of Greedy Pig, and the resulting changes in optimal strategy, can be explored. For instance, you could cap the number of rounds to be played. A pair of dice could be used, scoring by either adding the total of both or taking their difference; this also allows for a range of elimination conditions: ending the game on a double, when either die is a 2, for a particular total etc. You could also vary the frequency with which players choose to gamble, such as commiting them to two throws of the die each round. But, particularly for younger children, beware of making the game too complicated at the expense of fun!

Based on my experiences running primary school workshops as part of the Science Communcation in Action scheme at the University of Edinburgh. Unfortunately, I do not know who deserves the original credit for this game.

This May, I’ll be travelling all the way to Canada for ANTS-VIII, the Eighth Algorithmic Number Theory Symposium; I’m tacking a couple of days holiday on the front as well, so should be good!

So it’s been over two months since a post; more attentive readers will have noticed that there was one, but now there isn’t. I’ve moved away from thinking about cryptography to generalising some number/graph theoretic results of my supervisor, concerning matrices with constrained eigenvalues. However, this creates a problem: unless I ‘blog every up and down of the research process (which could be interesting, but would slow me down!) information on here becomes decreasingly accurate or relevant as I revise my thinking on the topic. Certainly it would be premature to present firm results at the moment.

But I can at least set the stage for more technically-minded readers (a friendlier explanation/illustration will hopefully follow once I truly understand all this!). Chris has characterised all symmetric integer matrices with the property that their eigenvalues are at most 2 in modulus; under a suitable transformation of their characteristic polynomials, these give cyclotomic polynomials and thus are referred to as cyclotomic matrices. Conveniently, any submatrix of a cyclotomic matrix is itself cyclotomic, so it suffices to find maximal examples. Although there are infinite families of these matrices, there are only a few ‘types’ possible.

These types are best understood by considering not the matrix, but an associated graph, where values in the matrix determine the weights on edges and nodes of the graph. This introduces a notion of equivalence, since many matrices will correspond to the same graph or certain well-defined variations on it. Further, we can adjoin nodes and edges to the corresponding graph to try and ‘grow’ towards maximal examples.

The motivation comes from finding polynomials of small Mahler measure- whilst a cyclotomic polynomial has measure 1, all others seem to be pushed away, with the smallest known value being 1.176… The question is how to generate small examples, and these matrices provide a way: by adjoining a single extra node to a maximal cyclotomic graph, a non-cyclotomic graph/matrix is obtained and thus a non-cyclotomic polynomial. The minimal graphs with this property (non-cyclotomic, but all subgraphs cyclotomic) often correspond to polynomials with some of the smallest known Mahler measures.

But some examples are not generated in this way, which is where I’ve stepped in. There is no reason to restrict attention to integer matrices, and I’ve established which imaginary quadratic extensions of the rationals give rise to rings of integers over which suitable matrices can be found. For a couple of fields, there are very few new (non rational-integer) cyclotomic matrices, and I have a complete description of them, but in others there are again infinite families as well as occasional examples that don’t generalise.

So I explore this behaviour by growing graphs/matrices, and try to spot patterns as they emerge from the fragments. I use the university’s parallel computing cluster Eddie for brute force work in SAGE, but such is the nature of the combinatorial explosion that even this doesn’t suffice without some mathematical insight along the way, as I try to refine my growing algorithms and capture equivalence as early as possible. I’m hopefully nearing the point where all examples fit into known families, at which point I’ll need to switch into serious mathematician mode and try and prove why this should be so. But for now I need to make sure that nothing unexpected tumbles out of each batch of calculations!

On a completely unrelated note, I’ve dragged modulo errors up to date with wordpress 2.5 and switched themes; please shout if you find I’ve broken something along the way.

I promised some of my tutorial students a demonstration of how the ‘highschool’ approach to Euclid’s algorithm can be reversed to give rise to the extended Euclidean algorithm – as opposed to the version in their lecture notes, which finds both gcd(a,b) and x,y such that ax+by=gcd(a,b) in one pass, at the price of some notational complexity. To do so, it seemed worth recapping some of the properties of divisibility that make Euclid’s algorithm tick, and give an application for its extended form. That ended up taking four pages, so I figured I’d post it here as well… you can read it behind the cut, or download the LaTeX-formatted pdf version.

Continue reading ‘The Extended Euclidean Algorithm’ »

The latest version of my ellnet class is ellnet2d_lowmem.spyx. It combines all the tricks I know of:

• The use of precomputed inverses for all steps, and precomputed squares/products for each step, as described by Stange,
• computation with a local vector to avoid overhead from function calls to keep the dictionary up-to-date,
• mixed block lengths as described in the previous post,
• and compilation to pyrex.

Thus it’s the fastest implementation I currently have for finding Tate pairings in SAGE (about twice as fast as accessing Stange’s PARI script from SAGE). Attach it in the normal way; example calculations are here.

(updated 10/i/08)

In an earlier post I described Stange’s algorithm for efficiently finding terms in elliptic nets (with a view to pairing computation). I also made the observation that a shorter block structure could be used for doubling- but once employed, it was not possible to perform a double-and-add. This meant that unless the desired term had a higher power of two as a factor then savings would be minor.

However, for a cost it is possible to ‘upgrade’ these short blocks to long blocks, since they contain enough information to recover the missing (k+4,0) term:

Better still, this only introduces an additional two multiplications and one inversion, since some of the terms feature in the precomputation (and assuming (2,0)² is computed once and stored for subsequent use):

Thus, given a short block centred at k, we can obtain the short block centred at 2k+1 (that is, perform a shortDoubleAdd). The dependencies are as follows:

Notice that a shortDoubleAdd is more expensive than DoubleAdd, even though it gives a short rather than long block! Thus a purely short-block algorithm would perform worse than the standard algorithm for binary strings with a high Hamming weight, since for each Double-and-add an inversion is introduced in place of a multiplication. However, when the Hamming weight is low, then the occasional cost of an inversion is balanced by the savings accrued during short doublings. To exploit this, whilst guarding against too many inversions, we introduce an algorithm that uses a mixture of standard (‘long’) and short blocks. Since only a single inversion is required to switch to long blocks, doing so allows us to more efficiently compute long runs of 1s in the binary representation by spreading the cost across several DoubleAdds.

We consider the generation of long or short blocks with centre 2k (double) or 2k + 1 (double-and-add) from long or short blocks of centre k. The cheapest such operation is the generation of the short block with centre 2k from a short or long block with centre k, at a cost of 31 multiplications, 6 squarings and no inversions. Using this as a base line, each procedure introduces the following additional operations:

We adopt a windowing approach with two-bit windows: that is, bit b_i informs whether we are to double or double-and-add, but b_i-1 is also examined to determine whether we should generate a long or short block.

• For b_ib_i-1=00, the short block approach clearly minimises the cost through these two bits.
• For b_ib_i-1=11, one should stay with long blocks if these are already in use, to avoid inversion. If short, adopting the long block immediately will mean only a single inversion is required for the following run of 1s.
• For b_ib_i-1=10, if short, then an inversion is required whether you go long or not: since being long is not necessary for the following double, we keep the multiplication count down by 2 by staying short. Similarly for long: no inversion is required to perform the Double-and-add for either length, but as the next operation will be a double, we go short to avoid the unnecessary 2 multiplications.
• For b_ib_i-1=01, then it is always worth staying short if you already are, defering the inversion until it is strictly required for b_i-1=1 (possibly choosing to go long then based on b_i-2). If currently long, going short will save 4 multiplications and a squaring (approximately 5 multiplications). Even if it proves necessary to upgrade to long for the very next bit, that will only cost around 3.6 multiplications (based on 1I=1.6M, see performance section). Thus even a single zero bit is worth going short for.

Hence the approach is to always go to (or stay with, if already the case) short blocks, unless b_ib_i-1=11 in which case one should go to (or stay with) long blocks. Thus a 2-bit window is sufficient to determine appropriate block length, leading to the following algorithm.

2-Bit Window Algorithm

Double-and-add Mixed-blocks Algorithm

INPUT: Integer n and long block centred at 1.

OUTPUT: Block centred at n.

1. Compute binary digits d_i of n such that n=(d_kd_k-1…d₁)₂ with d_k=1.
2. Set c=1 (centre), Set status=’long’
3. For i=k-1 down to 2 do:
   • If status=’long’
     • If d_i=1
       • If d_{i-1}=1 Compute block with centre 2c+1 via DoubleAddLongFromLong; Set c to 2c+1.
       • Else Compute short block with centre 2c+1 via DoubleAddShortFromLong; Set c to 2c+1; Set status=’short’.
     • Else
       Compute short block with centre 2c via DoubleShortFromLong; Set c to 2c; Set status=’short’.
   • Else
     • If d_i=1
       • If d_{i-1}=1 Compute block with centre 2c+1 via DoubleAddLongFromShort; Set c to 2c+1; Set status=’long’.
       • Else Compute block with centre 2c+1 via DoubleAddShortFromShort; Set c to 2c+1.
     • Else
       Compute short block with centre 2c via DoubleShortFromShort; Set c to 2c.
4. • If d_1=1
     • If status=’short’ Compute block with centre 2c+1 via DoubleAddShortFromShort.
     • Else Compute block with centre 2c+1 via DoubleAddShortFromLong.
   • Else Compute short block with centre 2c via DoubleShortFromShort.
5. Return final block.

Performance

As described before, the maximum possible gain is when n is a power of two, in which case the algorithm proceeds entirely by short doubles. In this case, there is a 12 percent reduction in the number of multiplications/squarings performed, with no inversions required.

Brute-force analysis of all possible 16-bit strings gives an average reduction of around 9 percent in the number of multiplications/squarings performed. Costing each inversion at 1.6 multiplications (based on average performance in SAGE for a 256-bit prime field), this leads to an average reduction of around 5 percent in the number of multiplications required for such strings. Testing several hundred 256-bit strings gives a similar figure.

Clearly, inversion is not viable if it will lead to a division-by-zero error. However, since the first zero along (i,0) will arise at (m,0), no such error will occur when performing Tate pairing computations.

A summary of this post and the earlier one on Stange’s algorithm is available as pdf, containing (hopefully) clearer copies of the dependency graphs.