The San Diego Unit Payoff Matrix
January 7, 2014
Last October I introduced the Payoff Matrix concept and showed results for six years of the La Jolla unit game based on the output generated by the Payoff Matrix software. Here I present similar results for the San Diego unit game and the combination of both unit games over the last two years.
It is most useful to look at regular partnerships that play in enough sessions to generate decent statistics. For the nearly two year period from the start of January 2012 through the end of November 2013, a 10 session minimum cut reduces the number of San Diego unit game partnerships to a manageable 37. Dave Oakley and Dave Walters lead the pack with a 57.55% average, which is very similar to their leading 56.96% average in the La Jolla unit game. They are followed by Randy Dougherty and Lynne Newman (57.35%), Steve Johnson and Mac Busby (56.62%), and Suzanne Lebending and Roger Doughman (55.75%). As noted in the La Jolla unit game payoff analysis, one can not state with certainty that any partnership is better than another but rather only state a degree of confidence that this is the case based on difference in average percentage and the error on each partnership’s average percentage. For example, Dave and Dave lead Randy and Lynne 57.55 ± 1.59% to 57.35 ± 1.30%. We must ask how confident we are that (57.55 − 57.35) = 0.20 ± 2.05% is greater than zero where the errors from each partnership have been added in quadrature. 0.20% is 0.097 standard deviations (0.20 / 2.05) away from zero. We can calculate the confidence using the error function; for example via 1 − erfc(0.097/sqrt(2)) / 2 in Matlab which yields 0.54. From this we can say with about 54% confidence that Dave and Dave are a stronger partnership than Randy and Lynne; in order words it is nearly a coin toss. Similarly we can say with about 64% confidence that Dave and Dave are stronger than Steve and Mac.
2012-2013 San Diego Unit Open Game Payoff Matrix
Both axes list the partnerships in order of decreasing strength. Each square shows how well the partnership listed on corresponding row does against the partnership in the corresponding column. The color scale runs from solid blue (30% or lower) to solid yellow (70% or better) with grey at 50%. White squares indicate partnerships that have never played any boards against each other. Pink squares indicate partnerships that can not interact because they have one or more players in common. The diagonal is always pink.
Move the mouse over the image to view the details for each square (matrix element). The hovering tooltip will show the names of the two partnerships, each partnership’s average against the field, the number of boards the partnerships have played against each other, and in bold the average results of those boards including an error. The average result shows how well the partnership on the row did against the partnership on the column. The matrix is anti-symmetric about the diagonal. Squares with a faint red X denote statistics based on relatively few boards, fewer than 10 in this case.
The upper right corner of the matrix is mostly yellow. This is because stronger partnerships usually have an advantage against weaker partnerships. Conversely the lower left corner is mostly blue.
The second matrix corrects for the difference in partnership skill, showing how much each partnership overperforms or underperforms against another partnership. The blue to yellow scale runs from -20% to 20% with grey at 0%. Move the mouse of the image to view the details for each square (matrix element). As before the partnership names and average percentage are shown. The over-perform / under-perform percentage is shown in bold. The ordinary percentage from the previous figure is also shown. Note: the previous figure also shows the over/under-performance statistics after ‘OU:’ as in ‘OU: 9.7%’.
Payoff tables and payoff graphs
The more yellowish and bluish square in the second matrix above are interesting because they indicate a pair that is strongly over/underperforming against another pair. The following table lists these relations in descending degree of payoff (or “exploitability”) where the pair-pair percentage is derived from at least 20 boards. Cnt is the number of boards that each interaction percentage based on.
| Overperforming Pair | Underperforming Pair | Cnt | OU% |
|---|---|---|---|
| Phyllis Pankow - Elizabeth Garber | Samuel Jordan - William Grant | 22 | 15.62 |
| Mary Huffaker - Ronald Huffaker | Suzanne LeBendig - Roger Doughman | 21 | 12.27 |
| Vicki Creamer - Jon Wright | Phyllis Pankow - Elizabeth Garber | 27 | 12.19 |
| Myrel Johnson - Gregory Wetz | Hanan Deeby - Bernard Figueiredo | 20 | 11.39 |
| Marty Bloomberg - Leila Bloomberg | Roy Green - Mary Green | 21 | 11.08 |
| Myrel Johnson - Gregory Wetz | Samuel Jordan - William Grant | 20 | 10.98 |
| Elaine Chan - Michael Mezin | Sue Kane - Sally Ishihara | 23 | 10.36 |
| Barb Holles - Stephanie Rake | Vicki Creamer - Jon Wright | 28 | 9.98 |
| Suzanne LeBendig - Roger Doughman | Hanan Deeby - Bernard Figueiredo | 28 | 9.17 |
| Vicki Creamer - Jon Wright | Ray Rowen - Alan Rowen | 22 | 9.10 |
| Phyllis Pankow - Elizabeth Garber | John Lagodimos - Kathie Angione | 36 | 8.71 |
| Sue Kane - Sally Ishihara | Evelyn Lantz - Fred Nimtz | 20 | 8.25 |
| Freda Anderson - Lynne O'Neill | Manoochehr Bahmanian - Lyle Kalish | 20 | 8.19 |
| Lena Jelusich - Alice Lane | Suzanne LeBendig - Roger Doughman | 30 | 7.69 |
| Andrew Loh - Patricia Loh | Lena Jelusich - Alice Lane | 20 | 7.27 |
| John Lagodimos - Kathie Angione | Barb Holles - Stephanie Rake | 29 | 7.21 |
| Mary Huffaker - Ronald Huffaker | Phyllis Pankow - Elizabeth Garber | 36 | 6.97 |
| Mary Huffaker - Ronald Huffaker | Lena Jelusich - Alice Lane | 44 | 6.89 |
| Evelyn Lantz - Fred Nimtz | Myrel Johnson - Gregory Wetz | 27 | 6.88 |
| Roy Green - Mary Green | Elaine Chan - Michael Mezin | 21 | 6.68 |
| John Lagodimos - Kathie Angione | Dan Dayani - Bette Cornelius | 20 | 6.58 |
| Manoochehr Bahmanian - Lyle Kalish | Lena Jelusich - Alice Lane | 25 | 6.50 |
| Suzanne LeBendig - Roger Doughman | Phyllis Pankow - Elizabeth Garber | 31 | 6.25 |
| Mary Huffaker - Ronald Huffaker | Freda Anderson - Lynne O'Neill | 37 | 6.04 |
| Barb Holles - Stephanie Rake | Elaine Chan - Michael Mezin | 29 | 5.96 |
| Marty Bloomberg - Leila Bloomberg | Sue Kane - Sally Ishihara | 21 | 5.87 |
| Lena Jelusich - Alice Lane | John Lagodimos - Kathie Angione | 35 | 5.85 |
| Myrel Johnson - Gregory Wetz | Mary Huffaker - Ronald Huffaker | 27 | 5.68 |
| Vicki Creamer - Jon Wright | Myrel Johnson - Gregory Wetz | 29 | 5.49 |
| Ray Rowen - Alan Rowen | Elaine Chan - Michael Mezin | 26 | 5.31 |
| Ray Rowen - Alan Rowen | Evelyn Lantz - Fred Nimtz | 21 | 5.13 |
| David Oakley - David Walters | Lena Jelusich - Alice Lane | 22 | 5.02 |
| Otto Newman - June Newman | Myrel Johnson - Gregory Wetz | 22 | 4.63 |
| Hanan Deeby - Bernard Figueiredo | Manoochehr Bahmanian - Lyle Kalish | 20 | 4.59 |
| Elaine Chan - Michael Mezin | Myrel Johnson - Gregory Wetz | 27 | 4.53 |
| Freda Anderson - Lynne O'Neill | John Lagodimos - Kathie Angione | 24 | 4.27 |
| Otto Newman - June Newman | Lena Jelusich - Alice Lane | 31 | 4.21 |
| Freda Anderson - Lynne O'Neill | Hanan Deeby - Bernard Figueiredo | 23 | 4.21 |
| Lena Jelusich - Alice Lane | Evelyn Lantz - Fred Nimtz | 29 | 4.13 |
| Freda Anderson - Lynne O'Neill | Otto Newman - June Newman | 25 | 4.01 |
| Mary Huffaker - Ronald Huffaker | Dan Dayani - Bette Cornelius | 24 | 3.86 |
| Myrel Johnson - Gregory Wetz | Marty Bloomberg - Leila Bloomberg | 26 | 3.78 |
| Ray Rowen - Alan Rowen | Mary Huffaker - Ronald Huffaker | 33 | 3.78 |
| Elaine Chan - Michael Mezin | Lena Jelusich - Alice Lane | 40 | 3.59 |
| Barb Holles - Stephanie Rake | Mary Huffaker - Ronald Huffaker | 50 | 3.53 |
| Hanan Deeby - Bernard Figueiredo | Samuel Jordan - William Grant | 23 | 3.31 |
| Barb Holles - Stephanie Rake | Marty Bloomberg - Leila Bloomberg | 32 | 3.19 |
| Manoochehr Bahmanian - Lyle Kalish | Myrel Johnson - Gregory Wetz | 24 | 3.03 |
The table gives us much to speculate about and yet it is still a lot to absorb. Part of the difficulty is that many pairs are listed multiple times. A graph may be a better representation. Here are two examples: OU ≥ 5% and OU ≥ 3%. The first graph represents the data in the table above down to advantages as small as 5% and the second represents the full table.
Unlike the Social Network graphs, the payoff graphs are directed, i.e they represent flow, the transport of over/underperformance around the partnership network. An incoming arrow represents a payoff to a partnership; an outgoing arrow represents payoff from a partnership.
It should be noted that although these are directed graphs, they are not directed acyclic graphs (DAGs), an important and well studied class of graphs. Cycles are easy to spot. For example Pankow-Garber pays off to Creamer-Wright who pay off to Holles-Rake, who payoff to Angione-Lagodimos, who complete the cycle by paying off to Pankow-Garber. So turns the wheel of exploitation.
Scientists and mathematicians may be concerned about the fact that inflow and outflow are not equal at each node (pair) given that the over/underperformance numbers have compensated for the difference in partnership ability. There are two reasons for this. First, only the most prominent payoffs are shown. Many smaller payoffs are suppressed in order to avoid total clutter. Second, the payoff calculations are based on different numbers of boards. For example, an underperformance of 3% over 40 boards would be balanced by a overperformance of 6% over 20 boards.
2012-2013 Combined La Jolla and San Diego Unit Game Payoff Matrix
There is considerable overlap between the players in the La Jolla unit and San Diego unit open games because one unit has its games on the first and third Sundays and the other has theirs on the second and forth Sundays of every month. It makes sense to combine the two data sets for better statistics. For this analysis we will only consider pairs that have played at least 20 session between the two games over the last two years.
The leaders are Randall Dougherty and Lynne Newman (57.49 ± 0.99%), David Oakley and Dave Walters (57.23 ± 1.10%), Steve Johnson and Mac Busby (55.96 ± 1.18%), Suzanne Lebendig and Roger Doughman (55.74 ± 0.67%), and Lynne O’Neill and William Grant (55.27 ± 0.75%). The more than 1% gap between the second and third place partnerships is quite significant. It means we can say with 78% confidence that Dave and Dave are better than Mac and Steve, up from the roughly 65% confidence based on either unit’s game results.
Here is the over/underperformance matrix.
Payoff tables and payoff graphs
The following table lists these relations in descending degree of payoff (or “exploitability”) where the pair-pair percentage is derived from at least 20 boards. Cnt is the number of boards that each interaction percentage based on.
| Overperforming Pair | Underperforming Pair | Cnt | OU% |
|---|---|---|---|
| Andrew Loh - Patricia Loh | Mary Huffaker - Ronald Huffaker | 20 | 17.84 |
| Phyllis Pankow - Elizabeth Garber | Samuel Jordan - William Grant | 22 | 15.51 |
| Freda Anderson - Gail Dunham | Alice Lane - Dorinda Lindvall | 22 | 14.36 |
| Alice Lane - Dorinda Lindvall | Marty Bloomberg - Leila Bloomberg | 21 | 13.61 |
| Kent Hartman - Maritha Pottenger | Roy Green - Mary Green | 23 | 13.22 |
| Vicki Creamer - Jon Wright | Phyllis Pankow - Elizabeth Garber | 27 | 12.37 |
| John Peter Lagodimos - Kathie Angione | Evelyn Lantz - Charlotte Blum | 21 | 11.82 |
| Myrel Johnson - Gregory Wetz | Hanan Deeby - Bernard Figueiredo | 20 | 11.53 |
| Elaine Chan - Michael Mezin | Sue Kane - Sally Ishihara | 28 | 11.16 |
| Myrel Johnson - Gregory Wetz | Samuel Jordan - William Grant | 20 | 10.98 |
| Barb Holles - Stephanie Rake | Vicki Creamer - Jon Wright | 28 | 9.90 |
| Phyllis Pankow - Elizabeth Garber | John Peter Lagodimos - Kathie Angione | 36 | 9.57 |
| Marty Bloomberg - Leila Bloomberg | Lynne Anderson - Ursula Kantor | 26 | 9.44 |
| Suzanne Lebendig - Roger Doughman | Marty Bloomberg - Leila Bloomberg | 20 | 9.26 |
| Marty Bloomberg - Leila Bloomberg | Carolyn Casey - Lena Jelusich | 26 | 9.11 |
| Lynne O'Neill - William Grant | Elaine Chan - Michael Mezin | 20 | 8.92 |
| Lynne Anderson - Ursula Kantor | Freda Anderson - Gail Dunham | 29 | 8.39 |
| Lena Jelusich - Alice Lane | Suzanne Lebendig - Roger Doughman | 34 | 8.36 |
| Charles Wilson Jr - Barbara Norman | Matthew Kidd - Patricia Lane | 20 | 8.32 |
| Ray Rowen - Alan Rowen | Freda Anderson - Gail Dunham | 20 | 8.24 |
| Mary Huffaker - Ronald Huffaker | Lynne Anderson - Ursula Kantor | 42 | 8.17 |
| Kent Hartman - Maritha Pottenger | Dan Dayani - Bette Cornelius | 30 | 8.06 |
| Andrew Loh - Patricia Loh | Lena Jelusich - Alice Lane | 22 | 7.94 |
| Freda Anderson - Lynne O'Neill | Manoochehr Bahmanian - Lyle Kalish | 20 | 7.85 |
| Kent Hartman - Maritha Pottenger | Ray Rowen - Alan Rowen | 34 | 7.84 |
| Phyllis Pankow - Betty-Jo Petersen | Carolyn Casey - Lena Jelusich | 20 | 7.67 |
| Evelyn Lantz - Fred Nimtz | Myrel Johnson - Gregory Wetz | 27 | 7.02 |
| Barb Holles - Stephanie Rake | Elaine Chan - Michael Mezin | 29 | 7.01 |
| Manoochehr Bahmanian - Lyle Kalish | Lena Jelusich - Alice Lane | 25 | 7.00 |
| Myrel Johnson - Gregory Wetz | Mary Huffaker - Ronald Huffaker | 27 | 6.95 |
| Suzanne Lebendig - Roger Doughman | Charles Wilson Jr - Barbara Norman | 22 | 6.94 |
| Phyllis Pankow - Betty-Jo Petersen | Alice Lane - Dorinda Lindvall | 21 | 6.89 |
| Dan Dayani - Bette Cornelius | Marty Bloomberg - Leila Bloomberg | 34 | 6.88 |
| Freda Anderson - Gail Dunham | John Peter Lagodimos - Kathie Angione | 22 | 6.81 |
| Charles Wilson Jr - Barbara Norman | Evelyn Lantz - Charlotte Blum | 29 | 6.74 |
| Charles Wilson Jr - Barbara Norman | Andrew Loh - Patricia Loh | 21 | 6.74 |
| Lena Jelusich - Alice Lane | John Peter Lagodimos - Kathie Angione | 35 | 6.65 |
| Lynne Anderson - Ursula Kantor | Elaine Chan - Michael Mezin | 27 | 6.62 |
| Evelyn Lantz - Charlotte Blum | Lynne Anderson - Ursula Kantor | 23 | 6.53 |
| John Peter Lagodimos - Kathie Angione | Dan Dayani - Bette Cornelius | 27 | 6.43 |
| Mary Huffaker - Ronald Huffaker | Lena Jelusich - Alice Lane | 48 | 6.42 |
| Suzanne Lebendig - Roger Doughman | Phyllis Pankow - Elizabeth Garber | 31 | 6.37 |
| Sue Kane - Sally Ishihara | Evelyn Lantz - Fred Nimtz | 23 | 6.35 |
| John Peter Lagodimos - Kathie Angione | Barb Holles - Stephanie Rake | 29 | 6.24 |
| Phyllis Pankow - Betty-Jo Petersen | Ray Rowen - Alan Rowen | 20 | 6.23 |
| John Peter Lagodimos - Kathie Angione | Otto Newman - June Newman | 31 | 6.20 |
| Otto Newman - June Newman | Lynne O'Neill - William Grant | 23 | 5.93 |
| Ray Rowen - Alan Rowen | Andrew Loh - Patricia Loh | 26 | 5.84 |
| Carolyn Casey - Lena Jelusich | Suzanne Lebendig - Roger Doughman | 22 | 5.82 |
| Mary Huffaker - Ronald Huffaker | Phyllis Pankow - Elizabeth Garber | 40 | 5.79 |
| Suzanne Lebendig - Roger Doughman | Kent Hartman - Maritha Pottenger | 42 | 5.71 |
| Vicki Creamer - Jon Wright | Myrel Johnson - Gregory Wetz | 29 | 5.56 |
| Phyllis Pankow - Betty-Jo Petersen | Lynne O'Neill - William Grant | 31 | 5.52 |
| David Oakley - David Walters | Lena Jelusich - Alice Lane | 22 | 5.51 |
| Dan Dayani - Bette Cornelius | Elaine Chan - Michael Mezin | 45 | 5.42 |
| Mary Huffaker - Ronald Huffaker | Lynne O'Neill - William Grant | 27 | 5.39 |
| Otto Newman - June Newman | Sue Kane - Sally Ishihara | 21 | 5.29 |
| Freda Anderson - Lynne O'Neill | John Peter Lagodimos - Kathie Angione | 24 | 5.24 |
| Roy Green - Mary Green | Otto Newman - June Newman | 35 | 5.22 |
| Samuel Jordan - Yoko Davis | Alice Lane - Dorinda Lindvall | 24 | 5.20 |
| Dan Dayani - Bette Cornelius | Suzanne Lebendig - Roger Doughman | 28 | 5.17 |
| Mary Huffaker - Ronald Huffaker | Suzanne Lebendig - Roger Doughman | 35 | 5.12 |
Here are corresponding graphs for OU ≥ 7% and OU ≥ 5%. The first graph represents the data in the table above down to advantages as small as 7% and the second represents the full table. Observe that the first graph is composed of three independent graphs, one as simple as the payoff from Kidd-Lane to Wilson-Norman.
Get the data
Download a zip file (3 MB) of all the tab delimited text files created by the Payoff Matrix software and used to generate the payoff matrix images and graphs above. For convenience, an Excel version of each file is also included. A filename such as U539-2012-2013-min-10-payoff-matrix.txt means the payoff matrix data for the 2012-2013 time period after restricting to partnerships that have played at least 10 sessions. If min-# is not included in the filename, no minimum session cut has been applied. The designation U526+U539 indicates data from combining the two unit games. The definition of each column is explained in the Payoff Matrix software documentation. The zip file also includes the images, the image specific JavaScript files, the Graphviz (.gvz) files, and the Scalable Vector Graphics files (.svg) for the graphs. A filename such as U539-2012-2013-mb-20-lp-3.gvz means results based on at least 20 boards (mb = “minimum boards”) and at least a 3% advantage (lp = “lowest percentage”).