Rigging the National Lottery (Is Hard)

As I’ve never won the National Lottery jackpot, I’m pretty sure the whole thing is rigged.

If you were in charge of taking peoples hard earned cash off of them and redistributing it back to a few of them, I bet you’d try and minimize the pay-out and pocket as much as you could. Now I’m not saying Camelot UK Lotteries Limited actually does this and The National Lottery is rigged, but you know, they definitely do and it is.

On draw nights, you can’t buy tickets between 7.30pm and 9.00pm before the winning balls are selected. Have you ever wondered what goes on for that hour and a half? I’ll tell you what; they are calculating which balls to select to give you the least money possible.

To minimize the pay-out you would need to compare all the possible combinations of six main balls plus the bonus ball, with all the entry lines which players have picked. Each comparison would check to see how many entry numbers matched the draw numbers, and the number of matches counted for all the entry lines. Finally, for each of the draw ball combinations, the total prize cost can be calculated using the stored matches count and the prize value for that number of matches.

Looking at the lotto prize breakdown from a past week (http://www.national-lottery.co.uk/player/lotto/results/prizeBreakdown.ftl):

Matches	No. of winners	£s per winner	Prize fund
Match 6	1	£7,959,312	£7,959,312
Match 5 plus Bonus	7	£68,525	£479,675
Match 5	176	£2,314	£407,264
Match 4	12,448	£173	£2,153,504
Match 3	256,646	£25	£6,416,150
Lotto Raffle	100	£20,000	£2,000,000
Totals	269,378		£19,415,905

The total prize fund was about £20,000,000. From the Lotto website (http://www.national-lottery.co.uk/player/p/goodcausesandwinners/wherethemoneygoes.ftl), 50% of the money spent goes to the prize fund, which means the total money spent buying tickets was about £40,000,000, which at £2 per ticket gives us:

£40,000,000 / £2 = 20,000,000 entries.

Even if we could compare one entry line to all combinations of draw numbers every second, this would still take:

20,000,000 seconds / (60 * 60 * 24) ~ 232 days

Way too long to process in 90 minutes.

To rig the lotto draw, we're going to need some serious speed. The following is an unoptimised implementation in Java to process ten randomly generated entry lines, to use as a base line benchmark:

Lotto1.java

package org.adrianwalker.lotto;

import java.util.Random;

public final class Lotto1 {

  /*
   * Estimated lotto prizes:
   *   https://www.national-lottery.co.uk/player/p/help/aboutlotto/prizecalculation.ftl
   */
  private static final int MATCH_6_PRIZE = 5000000;
  private static final int MATCH_5_PLUS_BONUS_PRIZE = 50000;
  private static final int MATCH_5_PRIZE = 1000;
  private static final int MATCH_4_PRIZE = 100;
  private static final int MATCH_3_PRIZE = 25;
  // match types
  private static final int MATCH_5_PLUS_BONUS = 7;
  private static final int MATCH_6 = 6;
  private static final int MATCH_5 = 5;
  private static final int MATCH_4 = 4;
  private static final int MATCH_3 = 3;
  // define lowest and highest ball numbers
  private static final int LOW_BALL = 1;
  private static final int HIGH_BALL = 49;
  // array sizes
  private static final int MAIN_BALLS_SIZE = 6;
  private static final int ENTRY_BALLS_SIZE = 6;
  private static final int ENTRIES_SIZE = 10;

  public static void main(final String[] args) {

    // lotto entries, could be read from a database or file?
    int[][] entries = new int[ENTRIES_SIZE][ENTRY_BALLS_SIZE];

    // draw balls for random entries
    int[] balls = {
      1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
      11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
      21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
      31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
      41, 42, 43, 44, 45, 46, 47, 48, 49
    };

    // create random entries for testing
    for (int i = 0; i < ENTRIES_SIZE; i++) {
      shuffle(balls);
      entries[i][0] = balls[0];
      entries[i][1] = balls[1];
      entries[i][2] = balls[2];
      entries[i][3] = balls[3];
      entries[i][4] = balls[4];
      entries[i][5] = balls[5];
    }

    long start = System.currentTimeMillis();

    // minimum values
    long minCost = Long.MAX_VALUE;
    int[] minCostMainBalls = new int[MAIN_BALLS_SIZE];
    int minCostBonusBall = -1;

    // maximum values
    long maxCost = Long.MIN_VALUE;
    int[] maxCostMainBalls = new int[MAIN_BALLS_SIZE];
    int maxCostBonusBall = -1;

    // iterate over main ball combinations
    int[] mainBalls = new int[MAIN_BALLS_SIZE];
    for (mainBalls[0] = LOW_BALL; mainBalls[0] <= HIGH_BALL; mainBalls[0]++) {
      for (mainBalls[1] = mainBalls[0] + 1; mainBalls[1] <= HIGH_BALL; mainBalls[1]++) {
        for (mainBalls[2] = mainBalls[1] + 1; mainBalls[2] <= HIGH_BALL; mainBalls[2]++) {
          for (mainBalls[3] = mainBalls[2] + 1; mainBalls[3] <= HIGH_BALL; mainBalls[3]++) {
            for (mainBalls[4] = mainBalls[3] + 1; mainBalls[4] <= HIGH_BALL; mainBalls[4]++) {
              for (mainBalls[5] = mainBalls[4] + 1; mainBalls[5] <= HIGH_BALL; mainBalls[5]++) {

                // iterate over bonus balls
                for (int bonusBall = LOW_BALL; bonusBall <= HIGH_BALL; bonusBall++) {

                  // skip bonus ball if its in the main balls
                  if (contains(mainBalls, bonusBall)) {
                    continue;
                  }

                  // init match counts to zero
                  int[] matchesCount = {0, 0, 0, 0, 0, 0, 0, 0};

                  // iterate over entries
                  for (int[] entry : entries) {
                    // count matches
                    int matches = matches(entry, mainBalls, bonusBall);
                    // increment matches for number of balls matched
                    matchesCount[matches]++;
                  }

                  // get total cost of draw
                  long cost = cost(matchesCount);

                  // keep track of highest/lowest draw costs
                  if (cost < minCost) {
                    minCost = cost;
                    minCostMainBalls = mainBalls.clone();
                    minCostBonusBall = bonusBall;
                  } else if (cost > maxCost) {
                    maxCost = cost;
                    maxCostMainBalls = mainBalls.clone();
                    maxCostBonusBall = bonusBall;
                  }
                }
              }
            }
          }
        }
      }
    }

    long end = System.currentTimeMillis();

    print(start, end,
            minCost, minCostMainBalls, minCostBonusBall,
            maxCost, maxCostMainBalls, maxCostBonusBall);
  }

  private static boolean contains(final int[] a, final int key) {

    for (int x : a) {
      if (x == key) {
        return true;
      }
    }

    return false;
  }

  private static void shuffle(final int[] a) {

    Random rand = new Random(System.currentTimeMillis());

    for (int i = 0; i < a.length; i++) {

      int index = rand.nextInt(i + 1);
      int x = a[index];
      a[index] = a[i];
      a[i] = x;
    }
  }

  private static int matches(final int[] entry, final int[] mainBalls, final int bonusBall) {

    int matches = 0;
    boolean bonusMatch = false;

    for (int entryNumber : entry) {
      for (int mainBall : mainBalls) {

        if (mainBall == entryNumber) {
          matches++;
        } else if (entryNumber == bonusBall) {
          bonusMatch = true;
        }
      }
    }

    if (matches == MATCH_5 && bonusMatch) {
      matches = MATCH_5_PLUS_BONUS;
    }

    return matches;
  }

  private static long cost(final int[] matchesCount) {

    long cost = 0;

    // add jackpot prize
    if (matchesCount[MATCH_6] > 0) {
      cost = MATCH_6_PRIZE;
    }

    // add lesser prizes, multiplied by number of winners
    cost += matchesCount[MATCH_5_PLUS_BONUS] * MATCH_5_PLUS_BONUS_PRIZE
            + matchesCount[MATCH_5] * MATCH_5_PRIZE
            + matchesCount[MATCH_4] * MATCH_4_PRIZE
            + matchesCount[MATCH_3] * MATCH_3_PRIZE;

    return cost;
  }

  private static void print(
          final long start, final long end,
          final long minCost, final int[] minCostMainBalls, final int minCostBonusBall,
          final long maxCost, final int[] maxCostMainBalls, final int maxCostBonusBall) {

    System.out.printf("\n--- entries ---\n");
    System.out.printf("random entries processed = %s\n", ENTRIES_SIZE);

    System.out.printf("\n--- time ---\n");
    System.out.printf("elapsed time = %s seconds\n", (end - start) / 1000);

    System.out.printf("\n--- min ---\n");
    System.out.printf("minimum cost = %s\n", minCost);
    System.out.printf("minimum main balls = %s,%s,%s,%s,%s,%s\n",
            minCostMainBalls[0],
            minCostMainBalls[1],
            minCostMainBalls[2],
            minCostMainBalls[3],
            minCostMainBalls[4],
            minCostMainBalls[5]);
    System.out.printf("minimum bonus ball = %s\n",
            minCostBonusBall == -1 ? "any" : minCostBonusBall);

    System.out.printf("\n--- max ---\n");
    System.out.printf("maximum cost = %s\n", maxCost);
    System.out.printf("maximum main balls = %s,%s,%s,%s,%s,%s\n",
            maxCostMainBalls[0],
            maxCostMainBalls[1],
            maxCostMainBalls[2],
            maxCostMainBalls[3],
            maxCostMainBalls[4],
            maxCostMainBalls[5]);
    System.out.printf("maximum bonus ball = %s\n",
            maxCostBonusBall == -1 ? "any" : maxCostBonusBall);
  }
}

On my very old L7500 1.60GHz laptop the output looks like this:

--- entries ---
random entries processed = 10

--- time ---
elapsed time = 636 seconds

--- min ---
minimum cost = 0
minimum main balls = 1,2,3,4,5,6
minimum bonus ball = 7

--- max ---
maximum cost = 5000000
maximum main balls = 1,6,8,15,22,37
maximum bonus ball = 2

636 seconds for ten entries! Way too slow! There are some optimisations we can perform to make this faster:

1. We only care about the bonus ball if we have 5 matching main balls - this loop can be executed only when we need it.
2. An intersection of arrays can be used to count the main ball array/entry array matches.
3. Sorted entry arrays can be used with a binary search to find bonus ball matches.

Lotto2.java

package org.adrianwalker.lotto;

import java.util.Arrays;
import java.util.Random;

public final class Lotto2 {

  /*
   * Estimated lotto prizes:
   *   https://www.national-lottery.co.uk/player/p/help/aboutlotto/prizecalculation.ftl
   */
  private static final int MATCH_6_PRIZE = 5000000;
  private static final int MATCH_5_PLUS_BONUS_PRIZE = 50000;
  private static final int MATCH_5_PRIZE = 1000;
  private static final int MATCH_4_PRIZE = 100;
  private static final int MATCH_3_PRIZE = 25;
  // match types
  private static final int MATCH_5_PLUS_BONUS = 7;
  private static final int MATCH_6 = 6;
  private static final int MATCH_5 = 5;
  private static final int MATCH_4 = 4;
  private static final int MATCH_3 = 3;
  // define lowest and highest ball numbers
  private static final int LOW_BALL = 1;
  private static final int HIGH_BALL = 49;
  // array sizes
  private static final int MAIN_BALLS_SIZE = 6;
  private static final int ENTRY_BALLS_SIZE = 6;
  private static final int ENTRIES_SIZE = 10;

  public static void main(final String[] args) {

    // lotto entries, could be read from a database or file?
    int[][] entries = new int[ENTRIES_SIZE][ENTRY_BALLS_SIZE];

    // draw balls for random entries
    int[] balls = {
      1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
      11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
      21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
      31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
      41, 42, 43, 44, 45, 46, 47, 48, 49
    };

    // create random entries for testing
    for (int i = 0; i < ENTRIES_SIZE; i++) {
      shuffle(balls);
      entries[i][0] = balls[0];
      entries[i][1] = balls[1];
      entries[i][2] = balls[2];
      entries[i][3] = balls[3];
      entries[i][4] = balls[4];
      entries[i][5] = balls[5];
      Arrays.sort(entries[i]);
    }

    long start = System.currentTimeMillis();

    // minimum values
    long minCost = Long.MAX_VALUE;
    int[] minCostMainBalls = new int[MAIN_BALLS_SIZE];
    int minCostBonusBall = -1;

    // maximum values
    long maxCost = Long.MIN_VALUE;
    int[] maxCostMainBalls = new int[MAIN_BALLS_SIZE];
    int maxCostBonusBall = -1;

    // iterate over main ball combinations
    int[] mainBalls = new int[MAIN_BALLS_SIZE];
    for (mainBalls[0] = LOW_BALL; mainBalls[0] <= HIGH_BALL; mainBalls[0]++) {
      for (mainBalls[1] = mainBalls[0] + 1; mainBalls[1] <= HIGH_BALL; mainBalls[1]++) {
        for (mainBalls[2] = mainBalls[1] + 1; mainBalls[2] <= HIGH_BALL; mainBalls[2]++) {
          for (mainBalls[3] = mainBalls[2] + 1; mainBalls[3] <= HIGH_BALL; mainBalls[3]++) {
            for (mainBalls[4] = mainBalls[3] + 1; mainBalls[4] <= HIGH_BALL; mainBalls[4]++) {
              for (mainBalls[5] = mainBalls[4] + 1; mainBalls[5] <= HIGH_BALL; mainBalls[5]++) {

                // init match counts to zero
                int[] matchesCount = {0, 0, 0, 0, 0, 0, 0, 0};

                // init bonus ball
                int bonusBall = -1;

                // iterate over entries
                for (int[] entry : entries) {

                  // count matches
                  int matches = intersection(entry, mainBalls);

                  // if 5 matches check the bonus ball
                  if (matches == MATCH_5) {

                    // iterate over bonus balls
                    for (bonusBall = LOW_BALL; bonusBall <= HIGH_BALL; bonusBall++) {

                      // skip bonus ball if its in the main balls
                      if (contains(mainBalls, bonusBall)) {
                        continue;
                      }

                      // break if the entries contain the bonus ball
                      if (contains(entry, bonusBall)) {
                        matches = MATCH_5_PLUS_BONUS;
                        break;
                      }
                    }
                  }

                  // increment matches for number of balls matched
                  matchesCount[matches]++;
                }

                // get total cost of draw
                long cost = cost(matchesCount);

                // keep track of highest/lowest draw costs
                if (cost < minCost) {
                  minCost = cost;
                  minCostMainBalls = mainBalls.clone();
                  minCostBonusBall = bonusBall;
                } else if (cost > maxCost) {
                  maxCost = cost;
                  maxCostMainBalls = mainBalls.clone();
                  maxCostBonusBall = bonusBall;
                }
              }
            }
          }
        }
      }
    }

    long end = System.currentTimeMillis();

    print(start, end,
            minCost, minCostMainBalls, minCostBonusBall,
            maxCost, maxCostMainBalls, maxCostBonusBall);
  }

  private static boolean contains(final int[] a, final int key) {

    return Arrays.binarySearch(a, key) != -1;
  }

  private static void shuffle(final int[] a) {

    Random rand = new Random(System.currentTimeMillis());

    for (int i = 0; i < a.length; i++) {

      int index = rand.nextInt(i + 1);
      int x = a[index];
      a[index] = a[i];
      a[i] = x;
    }
  }

  private static int intersection(final int[] a1, final int[] a2) {

    int count = 0;
    int i = 0;
    int j = 0;
    int a1length = a1.length;
    int a2length = a2.length;

    while (i < a1length && j < a2length) {
      if (a1[i] == a2[j]) {
        count++;
        i++;
        j++;
      } else if (a1[i] < a2[j]) {
        i++;
      } else if (a1[i] > a2[j]) {
        j++;
      }
    }

    return count;
  }

  private static long cost(final int[] matchesCount) {

    long cost = 0;

    // add jackpot prize
    if (matchesCount[MATCH_6] > 0) {
      cost = MATCH_6_PRIZE;
    }

    // add lesser prizes, multiplied by number of winners
    cost += matchesCount[MATCH_5_PLUS_BONUS] * MATCH_5_PLUS_BONUS_PRIZE
            + matchesCount[MATCH_5] * MATCH_5_PRIZE
            + matchesCount[MATCH_4] * MATCH_4_PRIZE
            + matchesCount[MATCH_3] * MATCH_3_PRIZE;

    return cost;
  }

  private static void print(
          final long start, final long end,
          final long minCost, final int[] minCostMainBalls, final int minCostBonusBall,
          final long maxCost, final int[] maxCostMainBalls, final int maxCostBonusBall) {

    System.out.printf("\n--- entries ---\n");
    System.out.printf("random entries processed = %s\n", ENTRIES_SIZE);

    System.out.printf("\n--- time ---\n");
    System.out.printf("elapsed time = %s seconds\n", (end - start) / 1000);

    System.out.printf("\n--- min ---\n");
    System.out.printf("minimum cost = %s\n", minCost);
    System.out.printf("minimum main balls = %s,%s,%s,%s,%s,%s\n",
            minCostMainBalls[0],
            minCostMainBalls[1],
            minCostMainBalls[2],
            minCostMainBalls[3],
            minCostMainBalls[4],
            minCostMainBalls[5]);
    System.out.printf("minimum bonus ball = %s\n",
            minCostBonusBall == -1 ? "any" : minCostBonusBall);

    System.out.printf("\n--- max ---\n");
    System.out.printf("maximum cost = %s\n", maxCost);
    System.out.printf("maximum main balls = %s,%s,%s,%s,%s,%s\n",
            maxCostMainBalls[0],
            maxCostMainBalls[1],
            maxCostMainBalls[2],
            maxCostMainBalls[3],
            maxCostMainBalls[4],
            maxCostMainBalls[5]);
    System.out.printf("maximum bonus ball = %s\n",
            maxCostBonusBall == -1 ? "any" : maxCostBonusBall);
  }
}

The output from this optimised version looks like this:

--- entries ---
random entries processed = 10

--- time ---
elapsed time = 17 seconds

--- min ---
minimum cost = 0
minimum main balls = 1,2,3,4,5,6
minimum bonus ball = any

--- max ---
maximum cost = 5000025
maximum main balls = 4,7,31,34,36,43
maximum bonus ball = any

17 seconds is a massive performance improvement over the previous code, but still not the sub second per entry speed we require.

The following code uses a long integer to represent the main balls as a bit mask, each 1 bit's position representing a main ball number. This bit mask can be used as a lookup to check entry numbers against:

Lotto3.java

package org.adrianwalker.lotto;

import java.util.Random;

public final class Lotto3 {

  /*
   * Estimated lotto prizes:
   *   https://www.national-lottery.co.uk/player/p/help/aboutlotto/prizecalculation.ftl
   */
  private static final int MATCH_6_PRIZE = 5000000;
  private static final int MATCH_5_PLUS_BONUS_PRIZE = 50000;
  private static final int MATCH_5_PRIZE = 1000;
  private static final int MATCH_4_PRIZE = 100;
  private static final int MATCH_3_PRIZE = 25;
  // match types
  private static final int MATCH_5_PLUS_BONUS = 7;
  private static final int MATCH_6 = 6;
  private static final int MATCH_5 = 5;
  private static final int MATCH_4 = 4;
  private static final int MATCH_3 = 3;
  // define lowest and highest ball numbers
  private static final int LOW_BALL = 1;
  private static final int HIGH_BALL = 49;
  // array sizes
  private static final int MAIN_BALLS_SIZE = 6;
  private static final int ENTRY_BALLS_SIZE = 6;
  private static final int ENTRIES_SIZE = 10;

  public static void main(final String[] args) {

    // lotto entries, could be read from a database or file?
    int[][] entries = new int[ENTRIES_SIZE][ENTRY_BALLS_SIZE];

    // draw balls for random entries
    int[] balls = {
      1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
      11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
      21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
      31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
      41, 42, 43, 44, 45, 46, 47, 48, 49
    };

    // create random entries for testing
    for (int i = 0; i < ENTRIES_SIZE; i++) {
      shuffle(balls);
      entries[i][0] = balls[0];
      entries[i][1] = balls[1];
      entries[i][2] = balls[2];
      entries[i][3] = balls[3];
      entries[i][4] = balls[4];
      entries[i][5] = balls[5];
    }

    long start = System.currentTimeMillis();

    // minimum values
    long minCost = Long.MAX_VALUE;
    int[] minCostMainBalls = new int[MAIN_BALLS_SIZE];
    int minCostBonusBall = -1;

    // maximum values
    long maxCost = Long.MIN_VALUE;
    int[] maxCostMainBalls = new int[MAIN_BALLS_SIZE];
    int maxCostBonusBall = -1;

    // iterate over main ball combinations
    int[] mainBalls = new int[MAIN_BALLS_SIZE];
    for (mainBalls[0] = LOW_BALL; mainBalls[0] <= HIGH_BALL; mainBalls[0]++) {
      for (mainBalls[1] = mainBalls[0] + 1; mainBalls[1] <= HIGH_BALL; mainBalls[1]++) {
        for (mainBalls[2] = mainBalls[1] + 1; mainBalls[2] <= HIGH_BALL; mainBalls[2]++) {
          for (mainBalls[3] = mainBalls[2] + 1; mainBalls[3] <= HIGH_BALL; mainBalls[3]++) {
            for (mainBalls[4] = mainBalls[3] + 1; mainBalls[4] <= HIGH_BALL; mainBalls[4]++) {
              for (mainBalls[5] = mainBalls[4] + 1; mainBalls[5] <= HIGH_BALL; mainBalls[5]++) {

                // create bitmask for match lookup
                long mainBallsBitmask = bitmask(mainBalls);

                // init match counts to zero
                int[] matchesCount = {0, 0, 0, 0, 0, 0, 0, 0};

                // init bonus ball
                int bonusBall = -1;

                // iterate over entries
                for (int[] entry : entries) {

                  // count matches
                  int matches = matches(entry, mainBallsBitmask);

                  // if 5 matches check the bonus ball
                  if (matches == MATCH_5) {

                    // iterate over bonus balls
                    for (bonusBall = LOW_BALL; bonusBall <= HIGH_BALL; bonusBall++) {

                      // skip bonus ball if its in the main balls
                      if (bitSet(mainBallsBitmask, bonusBall)) {
                        continue;
                      }

                      long entryBitmask = bitmask(entry);

                      // break if the entries contain the bonus ball
                      if (bitSet(entryBitmask, bonusBall)) {
                        matches = MATCH_5_PLUS_BONUS;
                        break;
                      }
                    }
                  }

                  // increment matches for number of balls matched
                  matchesCount[matches]++;
                }

                // get total cost of draw
                long cost = cost(matchesCount);

                // keep track of highest/lowest draw costs
                if (cost < minCost) {
                  minCost = cost;
                  minCostMainBalls = mainBalls.clone();
                  minCostBonusBall = bonusBall;
                } else if (cost > maxCost) {
                  maxCost = cost;
                  maxCostMainBalls = mainBalls.clone();
                  maxCostBonusBall = bonusBall;
                }
              }
            }
          }
        }
      }
    }

    long end = System.currentTimeMillis();

    print(start, end,
            minCost, minCostMainBalls, minCostBonusBall,
            maxCost, maxCostMainBalls, maxCostBonusBall);
  }

  private static void shuffle(final int[] a) {

    Random rand = new Random(System.currentTimeMillis());

    for (int i = 0; i < a.length; i++) {

      int index = rand.nextInt(i + 1);
      int x = a[index];
      a[index] = a[i];
      a[i] = x;
    }
  }

  private static boolean bitSet(final long bitMask, final int i) {
    return (bitMask & (1L << i)) > 0;
  }

  private static long setBit(final long bitmask, final int i) {
    return bitmask | (1L << i);
  }

  private static long bitmask(final int[] a) {

    long bitmask = 0;

    for (int i : a) {
      bitmask = setBit(bitmask, i);
    }

    return bitmask;
  }

  private static int matches(final int[] entry, final long mainBallsBitmask) {

    int matches = 0;

    for (int number : entry) {

      if (bitSet(mainBallsBitmask, number)) {
        matches++;
      }
    }

    return matches;
  }

  private static long cost(final int[] matchesCount) {

    long cost = 0;

    // add jackpot prize
    if (matchesCount[MATCH_6] > 0) {
      cost = MATCH_6_PRIZE;
    }

    // add lesser prizes, multiplied by number of winners
    cost += matchesCount[MATCH_5_PLUS_BONUS] * MATCH_5_PLUS_BONUS_PRIZE
            + matchesCount[MATCH_5] * MATCH_5_PRIZE
            + matchesCount[MATCH_4] * MATCH_4_PRIZE
            + matchesCount[MATCH_3] * MATCH_3_PRIZE;

    return cost;
  }

  private static void print(
          final long start, final long end,
          final long minCost, final int[] minCostMainBalls, final int minCostBonusBall,
          final long maxCost, final int[] maxCostMainBalls, final int maxCostBonusBall) {

    System.out.printf("\n--- entries ---\n");
    System.out.printf("random entries processed = %s\n", ENTRIES_SIZE);

    System.out.printf("\n--- time ---\n");
    System.out.printf("elapsed time = %s seconds\n", (end - start) / 1000);

    System.out.printf("\n--- min ---\n");
    System.out.printf("minimum cost = %s\n", minCost);
    System.out.printf("minimum main balls = %s,%s,%s,%s,%s,%s\n",
            minCostMainBalls[0],
            minCostMainBalls[1],
            minCostMainBalls[2],
            minCostMainBalls[3],
            minCostMainBalls[4],
            minCostMainBalls[5]);
    System.out.printf("minimum bonus ball = %s\n",
            minCostBonusBall == -1 ? "any" : minCostBonusBall);

    System.out.printf("\n--- max ---\n");
    System.out.printf("maximum cost = %s\n", maxCost);
    System.out.printf("maximum main balls = %s,%s,%s,%s,%s,%s\n",
            maxCostMainBalls[0],
            maxCostMainBalls[1],
            maxCostMainBalls[2],
            maxCostMainBalls[3],
            maxCostMainBalls[4],
            maxCostMainBalls[5]);
    System.out.printf("maximum bonus ball = %s\n",
            maxCostBonusBall == -1 ? "any" : maxCostBonusBall);
  }
}

The output from the bit mask optimised version looks like this:

--- entries ---
random entries processed = 10

--- time ---
elapsed time = 4 seconds

--- min ---
minimum cost = 0
minimum main balls = 1,2,4,5,6,8
minimum bonus ball = any

--- max ---
maximum cost = 5000025
maximum main balls = 2,20,27,43,46,47
maximum bonus ball = any

Four seconds for ten entries is approaching the sub second per entry performance we would need.

Obviously a faster algorithm is preferable to throwing more CPU at a problem, but porting the above Java code to C allows us to more effectively use the CPU we have, and gives us some speed improvement for free:

lotto.c

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <limits.h>
#include <time.h>

#define FALSE 0
#define TRUE !FALSE
#define SIZEOF(a) (sizeof a / sizeof a[0])

/*
 * Estimated lotto prizes:
 *   https://www.national-lottery.co.uk/player/p/help/aboutlotto/prizecalculation.ftl
 */
#define MATCH_6_PRIZE 5000000
#define MATCH_5_PLUS_BONUS_PRIZE 50000
#define MATCH_5_PRIZE 1000
#define MATCH_4_PRIZE 100
#define MATCH_3_PRIZE 25
// match types
#define MATCH_5_PLUS_BONUS 7
#define MATCH_6 6
#define MATCH_5 5
#define MATCH_4 4
#define MATCH_3 3
// define lowest and highest ball numbers
#define LOW_BALL 1
#define HIGH_BALL 49
// array sizes
#define MAIN_BALLS_SIZE 6
#define ENTRY_BALLS_SIZE 6
#define MATCHES_COUNT_SIZE 8
#define ENTRIES_SIZE 10

void shuffle(int a[], int size);
int bitSet(long bitMask, int i);
long setBit(long bitmask, int i);
long bitmask(int a[], int size);
int matches(int entry[ENTRY_BALLS_SIZE], long mainBallsBitmask);
long cost(int matchesCount[MATCHES_COUNT_SIZE]);
void print(
        time_t start, time_t end,
        long minCost, int minCostMainBalls[MAIN_BALLS_SIZE], int minCostBonusBall,
        long maxCost, int maxCostMainBalls[MAIN_BALLS_SIZE], int maxCostBonusBall);

int main(int argc, char** argv) {

  // lotto entries, could be read from a database or file?
  int entries[ENTRIES_SIZE][ENTRY_BALLS_SIZE];

  // draw balls for random entries
  int balls[] = {
    1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
    11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
    21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
    31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
    41, 42, 43, 44, 45, 46, 47, 48, 49
  };

  // create random entries for testing
  int i;
  for (i = 0; i < ENTRIES_SIZE; i++) {
    shuffle(balls, SIZEOF(balls));
    entries[i][0] = balls[0];
    entries[i][1] = balls[1];
    entries[i][2] = balls[2];
    entries[i][3] = balls[3];
    entries[i][4] = balls[4];
    entries[i][5] = balls[5];
  }

  time_t start = time(NULL);

  // minimum values
  long minCost = LONG_MAX;
  int minCostMainBalls[MAIN_BALLS_SIZE];
  int minCostBonusBall = -1;

  // maximum values
  long maxCost = LONG_MIN;
  int maxCostMainBalls[MAIN_BALLS_SIZE];
  int maxCostBonusBall = -1;

  // iterate over main ball combinations
  int mainBalls[MAIN_BALLS_SIZE];
  for (mainBalls[0] = LOW_BALL; mainBalls[0] <= HIGH_BALL; mainBalls[0]++) {
    for (mainBalls[1] = mainBalls[0] + 1; mainBalls[1] <= HIGH_BALL; mainBalls[1]++) {
      for (mainBalls[2] = mainBalls[1] + 1; mainBalls[2] <= HIGH_BALL; mainBalls[2]++) {
        for (mainBalls[3] = mainBalls[2] + 1; mainBalls[3] <= HIGH_BALL; mainBalls[3]++) {
          for (mainBalls[4] = mainBalls[3] + 1; mainBalls[4] <= HIGH_BALL; mainBalls[4]++) {
            for (mainBalls[5] = mainBalls[4] + 1; mainBalls[5] <= HIGH_BALL; mainBalls[5]++) {

              // create bitmask for match lookup
              long mainBallsBitmask = bitmask(mainBalls, SIZEOF(mainBalls));

              // init match counts to zero
              int matchesCount[MATCHES_COUNT_SIZE] = {0, 0, 0, 0, 0, 0, 0, 0};

              //init bonus ball
              int bonusBall = -1;

              // iterate over entries
              int i;
              for (i = 0; i < ENTRIES_SIZE; i++) {

                // count matches
                int m = matches(entries[i], mainBallsBitmask);

                // if 5 matches check the bonus ball
                if (m == MATCH_5) {

                  // iterate over bonus balls
                  for (bonusBall = LOW_BALL; bonusBall <= HIGH_BALL; bonusBall++) {

                    // skip bonus ball if its in the main balls
                    if (bitSet(mainBallsBitmask, bonusBall)) {
                      continue;
                    }

                    long entryBitmask = bitmask(entries[i], SIZEOF(entries));

                    // break if the entries contain the bonus ball
                    if (bitSet(entryBitmask, bonusBall)) {
                      m = MATCH_5_PLUS_BONUS;
                      break;
                    }
                  }
                }

                // increment matches for number of balls matched
                matchesCount[m]++;
              }

              // get total cost of draw
              long c = cost(matchesCount);

              // keep track of highest/lowest draw costs
              if (c < minCost) {
                minCost = c;
                memcpy(minCostMainBalls, mainBalls, MAIN_BALLS_SIZE * sizeof (int));
                minCostBonusBall = bonusBall;
              } else if (c > maxCost) {
                maxCost = c;
                memcpy(maxCostMainBalls, mainBalls, MAIN_BALLS_SIZE * sizeof (int));
                maxCostBonusBall = bonusBall;
              }
            }
          }
        }
      }
    }
  }

  time_t end = time(NULL);

  print(start, end, minCost, minCostMainBalls, minCostBonusBall,
          maxCost, maxCostMainBalls, maxCostBonusBall);

  return (EXIT_SUCCESS);
}

void shuffle(int a[], int size) {
  srand(time(NULL));
  int i;
  for (i = 0; i < size; i++) {
    int index = rand() % (i + 1);
    int x = a[index];
    a[index] = a[i];
    a[i] = x;
  }
}

int bitSet(long bitMask, int i) {
  return (bitMask & (1L << i)) > 0;
}

long setBit(long bitmask, int i) {
  return bitmask | (1L << i);
}

long bitmask(int a[], int size) {

  long bitmask = 0;

  int i;
  for (i = 0; i < size; i++) {
    bitmask = setBit(bitmask, a[i]);
  }

  return bitmask;
}

int matches(int entry[ENTRY_BALLS_SIZE], long mainBallsBitmask) {

  int matches = 0;

  int i;
  for (i = 0; i < ENTRY_BALLS_SIZE; i++) {

    if (bitSet(mainBallsBitmask, entry[i])) {
      matches++;
    }
  }

  return matches;
}

long cost(int matchesCount[MATCHES_COUNT_SIZE]) {

  long cost = 0;

  // add jackpot prize
  if (matchesCount[MATCH_6] > 0) {
    cost = MATCH_6_PRIZE;
  }

  // add lesser prizes, multiplied by number of winners
  cost += matchesCount[MATCH_5_PLUS_BONUS] * MATCH_5_PLUS_BONUS_PRIZE
          + matchesCount[MATCH_5] * MATCH_5_PRIZE
          + matchesCount[MATCH_4] * MATCH_4_PRIZE
          + matchesCount[MATCH_3] * MATCH_3_PRIZE;

  return cost;
}

void print(
        time_t start, time_t end,
        long minCost, int minCostMainBalls[MAIN_BALLS_SIZE], int minCostBonusBall,
        long maxCost, int maxCostMainBalls[MAIN_BALLS_SIZE], int maxCostBonusBall) {

  printf("\n--- entries ---\n");
  printf("random entries processed = %d\n", ENTRIES_SIZE);

  printf("\n--- time ---\n");
  printf("elapsed time = %lu seconds\n", (end - start));

  printf("\n--- min ---\n");
  printf("minimum cost = %lu\n", minCost);
  printf("minimum main balls = %d,%d,%d,%d,%d,%d\n",
          minCostMainBalls[0],
          minCostMainBalls[1],
          minCostMainBalls[2],
          minCostMainBalls[3],
          minCostMainBalls[4],
          minCostMainBalls[5]);
  printf("minimum bonus ball = %d\n", minCostBonusBall);

  printf("\n--- max ---\n");
  printf("maximum cost = %lu\n", maxCost);
  printf("maximum main balls = %d,%d,%d,%d,%d,%d\n",
          maxCostMainBalls[0],
          maxCostMainBalls[1],
          maxCostMainBalls[2],
          maxCostMainBalls[3],
          maxCostMainBalls[4],
          maxCostMainBalls[5]);
  printf("maximum bonus ball = %d\n", maxCostBonusBall);
}

Output from the above C code:

--- entries ---
random entries processed = 10

--- time ---
elapsed time = 2 seconds

--- min ---
minimum cost = 0
minimum main balls = 1,2,3,4,5,9
minimum bonus ball = -1

--- max ---
maximum cost = 5000050
maximum main balls = 1,4,7,10,21,30
maximum bonus ball = -1

Twice as fast as the Java version for ten entries! Lets up the number of entries to 1000 to get a better idea of performance:

--- entries ---
random entries processed = 1000

--- time ---
elapsed time = 155 seconds

--- min ---
minimum cost = 0
minimum main balls = 1,2,7,9,10,11
minimum bonus ball = -1

--- max ---
maximum cost = 6101000
maximum main balls = 3,4,5,32,34,36
maximum bonus ball = 1

Processing 20,000,000 total entry lines should take 20,000 times longer than the previous output:

155 seconds * 20,000 = 3,100,000 seconds

3,100,000 seconds / (60 * 60 * 24) ~ 36 days

Even with the code optimisations and porting to C, 36 days is way longer than our 90 minute window.

Rigging the National Lottery is hard.

There will be further optimisations to be had with this code, but who cares, we all know how the Lottery is really rigged...

... MAGNETS!

Source Code

• Java and C code available in GitHub - lotto

Learning JavaScriptMVC Review

JavaScriptMVC is an open-source Rich Internet Application framework based on jQuery and OpenAjax. It extends those libraries with a model–view–controller architecture and tools for testing and deployment. As it does not depend on server components, it can be combined with any web service interface and server-side language.

Packt Publishing requested that I review their new book on the subject, Learning JavaScriptMVC by Wojciech Bednarski, available to buy from Packt's website here:
http://link.packtpub.com/hFZPlQ

The book starts with a preface which points out that the JavaScriptMVC framework is a client side only technology and gives an overview of the six chapters and what they cover, as well as the usual intended audience and conventions used in the book.

Chapter one starts with an introduction to JavaScriptMVC, giving details of its constituent parts: StealJS, FuncUnit, jQueryMX and DocumentJS. The chapter moves on to describe the advantages of using the JavaScriptMVC framework over other alternatives, and spends some time discussing the advantages and disadvantages of different web application architectures.

With the what and why of JavaScriptMVC laid out the author goes on to offer three methods of getting up and running with the framework:

1. Download the complete package from the official website.
2. Pull code and libraries from the authors GitHub.
3. Use Vagrant to download and configure a ready to go virtual machine.

The author discourages using the first method for anything other than a fast try-out, recommends the second method for actual deployment, and advocates the third method for development as it provides an encapsulated development environment.

I'd never used Vagrant before, and liked the idea of a preconfigured and encapsulated dev environment, so I was exited to try out the third method. In retrospect I should have been suspicious of any method which requires you to download literally gigabytes of software to use a JavaScript framework.

Installing Vagrant and it's dependencies on my Linux Mint machine was pretty straight forward, just using:

apt-get install vagrant

VirtualBox is a dependency of Vagrant which was also installed. Next I downloaded the specified Vagrant configuration file and issued the command to fire up the preconfigured server and dev environment, very exciting:

vagrant up

Hundreds and hundreds of megabytes of downloads later and Vagrant reports that the server cannot start because VirtualBox required me to make changes to my BIOS settings.

Lets just think about this for a minute, installation of multiple software packages, a 1.3 GB virtual machine image download and a BIOS setting change just to run a client side JavaScript framework – and this is the recommended method to get started - absolute madness.

I persisted with the Vagrant method and continued to follow the instructions in chapter one to build a Todo list, until I noticed that changes I was making to the pages HTML and JavaScript files were not being reflected in the browser when refreshed.

After some Googling, I found this explanation and solution - clear the nginx cache in Vagrant:

http://jeremyfelt.com/code/2013/01/08/clear-nginx-cache-in-vagrant

The commands I executed in addition to the above information:-

vagrant ssh
vagrant@lucid64:~$ sudo vi /etc/nginx/nginx.conf
vagrant@lucid64:~$ sudo service nginx restart


Finally after getting a working dev environment I typed out the code in the chapter to get the basic foundation of the application. I fired up my browser to the URL specified and it didn't work.

I'd already wasted too much time on this method and decided to abandon Vagrant, and try method two - get the files from GitHub.

I quickly installed Apache web server on my machine and cloned the authors GitHib repository. I moved my source files and reloaded the project in my browser. It didn't work again.

Last but not least I decided to try the first installation method described by the author - downloading the complete JavaScriptMVC package from the official website. This method gave me an installation of JavaScriptMVC which worked. I would recommend this method.

I completed chapter one, typing out the remaining code for the application, only to find myself with an app which did nothing and reported no errors.

I also took the downloadable source code for the chapter provided by the author and ran it, only to achieve the same result.

At this point I have given up on the book, I can't waste any more time trying to get the code to work. There is no point in learning how to document and test an application which doesn't work.

This book has made me frustrated and angry and I couldn't recommend it any less. I can only assume this book is a cruel joke by the author make me feel stupid and unproductive. Hopefully I can still get some use from this book, come winter I will be using it as a £14.99 fire lighter.

You may find other reviews of this book less ranty:

• http://marxsoftware.blogspot.co.uk/2013/06/book-review-learning-javascriptmvc.html
• http://www.amazon.co.uk/product-reviews/B00BN4P89A

Downloads From Packt

• Sample Chapter - 9781782160205_Chapter_02.pdf
• Example Code - Learning JavaScriptMVC.zip

Selected Blog Posts Sources In GitHub

The source code for some of my most viewed posts is available in GitHub here:-

github.com/adrianwalker

Projects currently available:-

Post	Repository
Properties design pattern and Prototype-based programming in Java	properties-pattern
HTML5 Video Pseudostreaming with Java 7	pseudostreaming
Java Multiline String	multiline-string
jboard - yet another imageboard	jboard
GWT File Upload With Event-Based Progress Bar	gwt-upload-progress
map, foldr & foldl higher-order functions in Java	functionals

Properties design pattern and Prototype-based programming in Java

After reading Steve Yegge's post about the properties design pattern and how it can be used to create a prototype-based object system, I thought I'd have a go at an implementation in Java to understand more.

If you haven't read the above post already, it definitely worth a read (or two), and makes everything below make more sense, give it a go.

First, I wanted to define the minimum API needed to get a working properties object as described in the Overview section. A basic properties interface:

Properties.java

package org.adrianwalker.propertiespattern;

interface Properties<N, V> {

  V get(N name);

  V put(N name, V value);

  boolean has(N name);

  V remove(N name);
}

I'll defer describing the implementation until the end, because as soon as you have the basic functionality done there is a load more you're going to want to add.

Once you've got your Properties implementation storing name/value pairs and inheriting from a prototype Properties object stored against a reserved key name, at that point, your going to want to add some way for your objects to simulate methods to manipulate the property values.

Because Java doesn't have first class functions, a methods functionality needs to be encapsulated using the Strategy pattern.

Strategy.java

package org.adrianwalker.propertiespattern;

interface Strategy<N, V, T, A> {

  T execute(Properties<N, V> properties, A... args);
}

In addition to the Strategy interface, I extended the basic Properties interface with an execute method to call a Strategy property:

ExecutableProperites.java

package org.adrianwalker.propertiespattern;

interface ExecutableProperites<N, V, T, A> extends Properties<N, V> {

  T execute(N name, A... args);
}

You're also probably going to want a way of returning an object's property names, possibly filtered with by a regex, or a globing pattern, or partial string match or some kind object comparison. So another extension to the Properties interface:

FilterableProperties.java

package org.adrianwalker.propertiespattern;

import java.util.Set;

interface FilterableProperties<N, V> extends Properties<N, V> {

  Set<N> filter(N filter);
}

And it would be good to have an interface for cloning prototype-based objects:

CloneableProperties.java

package org.adrianwalker.propertiespattern;

interface CloneableProperties<N, V> extends Properties<N, V>, Cloneable {

  CloneableProperties<N, V> clone();
}

Implementation

With all of that out of the way it's time for an implementation. This implementation stores property name/value pairs in a HashMap, and filters using regular expressions:

HashMapProperties.java

package org.adrianwalker.propertiespattern;

import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.regex.Pattern;

public final class HashMapProperties<V, T, A> implements
        Properties<String, V>,
        ExecutableProperites<String, V, T, A>,
        CloneableProperties<String, V>,
        FilterableProperties<String, V> {

  public static final String PROTOTYPE = "prototype";
  private final Map<String, V> properties;

  public HashMapProperties() {

    this.properties = Collections.synchronizedMap(new HashMap());
  }

  private HashMapProperties(final Map<String, V> map) {

    this.properties = Collections.synchronizedMap(new HashMap<String, V>(map));
  }

  @Override
  public V get(final String name) {

    if (properties.containsKey(name)) {

      return properties.get(name);
    }

    if (properties.containsKey(PROTOTYPE)) {

      Properties<String, V> prototype =
              (Properties<String, V>) properties.get(PROTOTYPE);

      return prototype.get(name);
    }

    return null;
  }

  @Override
  public V put(final String name, final V value) {

    if (PROTOTYPE.equals(name) && !(value instanceof Properties)) {

      throw new IllegalArgumentException(
              "prototype must be an instance of Properties");
    }

    return properties.put(name, value);
  }

  @Override
  public boolean has(final String name) {

    if (properties.containsKey(name)) {

      return null != properties.get(name);
    }

    if (properties.containsKey(PROTOTYPE)) {

      Properties<String, V> prototype =
              (Properties<String, V>) properties.get(PROTOTYPE);

      return prototype.has(name);
    }

    return false;
  }

  @Override
  public V remove(final String name) {

    if (properties.containsKey(PROTOTYPE)) {

      Properties<String, V> prototype =
              (Properties<String, V>) properties.get(PROTOTYPE);

      if (prototype.has(name)) {

        properties.put(name, null);

        return prototype.get(name);
      }
    }

    if (properties.containsKey(name)) {

      return properties.remove(name);
    }

    return null;
  }

  @Override
  public HashMapProperties<V, T, A> clone() {

    return new HashMapProperties<V, T, A>(properties);
  }

  @Override
  public T execute(final String name, final A... args) {

    V property = get(name);

    if (property instanceof Strategy) {

      return ((Strategy<String, V, T, A>) property).execute(this, args);
    }

    return null;
  }

  @Override
  public Set<String> filter(final String regex) {

    Pattern pattern = Pattern.compile(regex);

    Set<String> filteredNames = new HashSet<String>();

    Set<String> names = properties.keySet();

    synchronized (properties) {
      for (String name : names) {
        if (!PROTOTYPE.equals(name) && pattern.matcher(name).matches()) {
          filteredNames.add(name);
        }
      }
    }

    if (properties.containsKey(PROTOTYPE)) {
      FilterableProperties<String, V> prototype =
              (FilterableProperties<String, V>) properties.get(PROTOTYPE);
      filteredNames.addAll(prototype.filter(regex));
    }

    return filteredNames;
  }
}

Example Usage

The following example shows how you might use the above implementation. A prototype Properties object is created with a 'greeting' property and a 'getGreeting' Strategy property. These properties are inherited by a new Properties object.

The Properties object is cloned and it's greeting property is overridden.

The cloned Properties object's properties are filtered to return the getter Strategy's property name, then this is executed for the Properties object and it's clone:

Example.java

package org.adrianwalker.propertiespattern;

import java.util.Set;
import static org.adrianwalker.propertiespattern.HashMapProperties.PROTOTYPE;

public final class Example {

  public static void main(final String[] args) {
 
    // create a prototype properties object with a greeting and a 
    // strategy which returns the greeting and an argument
 
    Properties prototype = new HashMapProperties();
    prototype.put("greeting", "Hello");
    prototype.put("getGreeting", new Strategy() {
      @Override
      public String execute(final Properties properties, final Object... args) {

        return String.format("%s %s", properties.get("greeting"), args[0]);
      }
    });
 
    // create a properties object which inherit the greeting and 
    // strategy from the prototype

    HashMapProperties properties = new HashMapProperties();
    properties.put(PROTOTYPE, prototype);

    // clone the properties object and override the greeting property

    HashMapProperties clone = properties.clone();
    clone.put("greeting", "Ahoy");

    // filter for the getter properties, then execute the getter
    // on each properties object with a message argument and
    // print the message

    Set<String> getters = clone.filter("get.*");
    for (String getter : getters) {
      System.out.println(properties.execute(getter, "World"));
      System.out.println(clone.execute(getter, "Sailor"));
    }
  }
}

The example above should print the output:

Hello World
Ahoy Sailor

Finally

I've not implemented half of the functionality mentioned in original post, and not a fraction of a fraction of the possible applications for this pattern. The above code and a set of unit tests are available for download at the bottom of this page.

If you need the level of flexibility to warrant using this pattern, then, to quote the original article "use a programming language better suited for implementing the pattern: ideally, one that supports it from the ground up". So expect some JavaScript related posts in the future.

Source Code

• Properties-pattern Maven Project - properties-pattern.zip

Usage

Compile the project with 'mvn clean install'.

Email Sky Router Public IP Address with Gmail in Python

As far as I know, Sky Broadband don't offer a static IP address. You're going to want a way of keeping track of your externally available dynamic IP, so you can access your machine from outside of your network.

The python script below scrapes your Sky router's status page for your current IP and emails it your Gmail account.

Remember to change the Gmail username and password variables from '????????' to your Gmail authentication credentials. Run this script periodically as a cron job, and as long as you have Gmail access you'll know your IP.

#!/usr/bin/python

import smtplib
from email.mime.text import MIMEText
import urllib2
import re

# router login page
router_login = "http://192.168.0.1"
# router status url
router_status = "http://192.168.0.1/sky_router_status.html"
# router logout page
router_logout = "http://192.168.0.1/sky_logout.html"
# default sky router username and password
router_username = "admin"
router_password = "sky"
# gmail email address
email = "????????@gmail.com"
# gmail password
email_password = "????????"
# gmail smtp settings
server = "smtp.gmail.com"
port = 587
# email subject
subject = "Server IP"

# login to router
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, router_login, router_username, router_password)
authhandler = urllib2.HTTPBasicAuthHandler(passman)
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
urllib2.urlopen(router_login)

# get the first ip on the status page
ip = urllib2.urlopen(router_status).read()
ip = re.findall(r"[0-9]+(?:\.[0-9]+){3}", ip)
ip = ip[0]

# logout of router
urllib2.urlopen(router_logout)

# check for last ip address
f = open(".last_ip", "a+")
f.seek(0, 0)
last_ip = f.readline()

# has ip changed
if ip != last_ip:

  # store the new ip
  f.truncate(0)
  f.write(ip)

  session = smtplib.SMTP(server, port)

  session.ehlo()
  session.starttls()
  session.login(email, email_password)

  body = "http://" + ip

  msg = MIMEText(body)
  msg['Subject'] = subject
  msg['From'] = email
  msg['To'] = email

  session.sendmail(email, email, msg.as_string())
  session.quit()

f.close()

Iterate over first n files in a directory with Java 6 and JNA on Linux

I needed to be able to iterate over the first n number of files in a directory, without bringing them all back at once using File.list(). This is simple enough to do in Java 7 using the new DirectoryStream interface, but I only had access to Java 6 for this project.

Similar functionality can be reproduced using calls to native operating system functions using Java Native Access.

This example works on Linux but could easily be adapted for Windows.

DirectoryStream.java

package org.adrianwalker;

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.Structure;
import java.io.Closeable;
import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.NoSuchElementException;

public final class DirectoryStream implements Iterable<File>, Closeable {

  private String dirName;
  private POSIX posix;
  private Pointer dirp;

  public DirectoryStream(final String dirName) {
    this.dirName = dirName;
    this.posix = (POSIX) Native.loadLibrary("c", POSIX.class);
  }

  @Override
  public Iterator<File> iterator() {

    return new Iterator<File>() {
      private boolean gotNext = false;
      private Dirent file;

      @Override
      public boolean hasNext() {

        if (!gotNext) {
          getNext();
        }

        return file != null;
      }

      @Override
      public File next() {

        if (!gotNext) {
          getNext();
        }

        if (null == file) {
          throw new NoSuchElementException();
        }

        gotNext = false;

        return new File(dirName, file.getName());
      }

      @Override
      public void remove() {
        throw new UnsupportedOperationException();
      }

      private void getNext() {
        if (null == dirp) {
          dirp = posix.opendir(dirName);
        }

        file = posix.readdir(dirp);

        gotNext = true;
      }
    };
  }

  @Override
  public void close() throws IOException {
    posix.closedir(dirp);
  }

  public interface POSIX extends Library {

    public Pointer opendir(String dirname);

    Dirent readdir(Pointer dirp);

    int closedir(Pointer dirp);
  }

  public static final class Dirent extends Structure {

    public long d_ino;
    public long d_off;
    public short d_reclen;
    public char d_type;
    public char[] d_name = new char[256];

    public String getName() {
      return getPointer().getString(8 + 8 + 2 + 1, false);
    }

    public long getInode() {
      return getPointer().getLong(0);
    }

    public void setUseMemory(Pointer p) {
      useMemory(p);
    }
  }
}

Example usage:-

Main.java

package org.adrianwalker;

import java.io.File;
import java.io.IOException;

public final class Main {

  private static final int MAX_FILES = 10;
  private static final String CURRENT_DIR = ".";
  private static final String PARENT_DIR = "..";

  public static void main(final String[] args) throws IOException {

    String dirName = "/tmp/files";
    DirectoryStream directoryStream = new DirectoryStream(dirName);

    int fileCount = 0;
    for (File file : directoryStream) {

      String name = file.getName();
      if (name.equals(CURRENT_DIR) || name.equals(PARENT_DIR)) {
        continue;
      }      
      
      if (fileCount++ == MAX_FILES) {
        break;
      }

      System.out.println(file);
    }
  }
}

Source Code

• JNA DirectoryStream Maven Project - jna-directorystream.zip

Usage

Build the project with 'mvn clean install' and run the Main class.

Java EE 6 Cookbook for Securing, Tuning, and Extending Enterprise Applications Review

Java Enterprise Edition is Oracle's enterprise Java computing platform. The platform provides an API and runtime environment for developing and running enterprise software, including network and web services. Java EE extends the Java Standard Edition providing an API for object-relational mapping, distributed and multi-tier architectures, and web services.

Packt Publishing requested that I review one of their titles about Java EE: Java EE 6 Cookbook for Securing, Tuning, and Extending Enterprise Applications by Mick Knutson, available to buy from Packt's website.

Java EE 6 Cookbook for Securing, Tuning, and Extending Enterprise Applications is an OK book, the breadth of topics covered is vast, but outside of the scope of the book at times. It feels like the book is trying to be an introductory tutorial to most aspects of Java development. Not that this is a bad thing, lots of the topics covered are important to day to day Java development and the advice given is pragmatic. Using the information as a starting point to expand your knowledge would make you a Java jack of all trades.

I found parts of it very informative and other parts frustrating, overall I'd say its worth a read but might not be what you expect. It not a book about JEE development as much as about peripheral concerns around JEE development which you might encounter in a project.

The information is presented in recipes - how-to examples focusing on a specific JEE feature, or other topic, put together to form a cookbook. Each recipe is presented in a formulaic way, with information under the sub-headings: 'Getting ready', 'How to do it...', 'How it works...' and 'There's more'.

A Java EE 6 installation, application server, and an editor are requirements to reproduce the examples in the book. GlassFish application server is used in a number of recipes, and some recipes are specific to the IntelliJ IDEA editor, but for the majority of the book any IDE and app server could be used.

The code to accompany the book available as a download isn't great. The Maven project pom.xml files needed altering before the code would build, and source code examples for some chapters seemed to be altogether missing.

Chapter one highlights key changes in Java EE 6 and how new features will improve performance and ease of development. The chapter first covers old JEE APIs, showing examples of what older technologies in old applications might need replacing, and what to replace those older technologies with. Covered next are introductions to the specifications and APIs including, Context and Dependency Injection (CDI), EJB 3.1, JPA 2.0, JAX-RS 1.1, Sevlet 3.0, WebBeans 1.0, and JSF 2.0.

Chapter two covers JPA persistence, some good recipes in this chapter are the 'Understanding @CollectionTable' recipe, covering the new annotation in the JPA 2.0 specification, and the JPA audit pattern in recipe 'Auditing historical JPA Operations'. The chapter finishes with an introduction to profiling JPA operations with the IntelliJ IDEA profile. I've never gotten on board with IntelliJ IDEA, and much prefer NetBeans and its profiler.

Chapter three covers JEE security, starting with definitions of security terms and deployment descriptors for role definitions and mapping in GlassFish. Next is a pointlessly detailed description of Java EE authentication methods complete with activity diagrams and HTML request/response headers.

XML based authorization configuration is introduced, then new programmatic and annotation based security in the next recipe.

A recipe I found useful later in the chapter was 'Configuring Linux firewall rules' - using iptables to configure access to SSH on Linux. Although I found it informative, I can't help thinking it doesn't really belong in this book - and the same with the next recipe 'Securely obfuscating Java byte-code', I fail to see how obfuscating code makes an application any more secure.

Chapter 4 deals with testing. Testing is clearly an important topic, and consideration of testing strategies, frameworks, tools and libraries should part of any Java project, but I was surprised to see this as a chapter in this book. It's more of a general software development topic which probably needs a book in its own right, I'm not sure it should be in a book about Securing, Tuning and Extending Enterprise Java apps.

The chapter starts with a good recipe demonstrating the use of the Maven Surefire plugin to enable a remote debugging session.

The rest of the chapter gives examples of testing frameworks and libraries, not just applicable to JEE testing but also to Java SE. JUnit and DBUnit are used automate database testing, along with test object mocking libraries Mockito, Powermock and Hamcrest.

Selenium and soapUI testing tools are introduced at the end of the chapter, with some good information on how to integrate these tools into your build and test process with Maven.

Chapter 5 uses the Groovy and Scala scripting languages running on the JVM, integrated with existing Java code, to write more concise unit test code. There is also a Jython servlet recipe demonstrating how to create a simple web page. Although not strictly JEE related, I enjoyed these recipes and can see how using them to write less code can have productivity increases.

The rest of the chapter deals with Aspect Orient Programming with AspectJ and CDI, and starts by describing some AOP terms: Cross-cutting-concerns, Advice, Pointcut, Aspect Joinpoint and Weaving. Recipes include weaving AspectJ advice into your own code and using AspectJ with existing 3rd party libraries, and using CDI interceptors. The AspectJ recipes in this chapter are not limited to JEE programming, and would be applicable to Java SE projects also.

Chapter 6 goes a totally off track with mobile development recipes. Recipes cover mobile web frameworks, native code generators, native web runtime frameworks, development considerations for iOS, Objective-C, mobile development IDEs, Android development, online emulators and other totally random mobile development things which should have been way out of the scope of this book!

Chapter 7 explores configuration and management, the important recipes from this chapter cover the use of VisualVM to remotely interact with and monitor Tomcat and GlassFish servers using JMX.

There is a recipe in this chapter on the use of JRebel for rapid redeployment. JRebel is a non-free, closed source product, although a free 14-day trial download is available. I try not to use non-free, closed source software, and could not promote the use of this product on my blog.

With Chapter 8 the book finishes with some useful tutorials for using profiling tools to monitor JVM memory usage and utilities to monitor HTTP activity. It covers the use of VisualVM and the jstatd plugin, monitoring TCP connections with Netstat, proxying connection with TCPMon to observe HTTP traffic, and server monitoring with Munin.

Unfortunately the chapter finishes with another recipe for a closed source, non-free product, HTTP Debugger. Again a 14 day trial download is available, but as the software is not FOSS I did not use it.

Overall I think there are some good tips in this book, but some chapters just feel like filler. If you're not experienced already with JEE development, then this isn't the book to start to learn with. If you are experienced then some chapters will seem a bit basic, and I'm sure some will seem a bit off topic. But all JEE developers will take at least a handful of useful recipes from this book.

This is an OK book but not what I was expecting. I'm not sure if the author is trying to take systems approach to JEE development, or just picked some of his favorite topics on the periphery of JEE development and put them together. I'd make sure you have a flick through it before you buy.

Downloads From Packt

• Sample Chapter - 9781849683166-Chapter-4.pdf
• Example Code - 3166_Code.zip