ACBLscore Game Files Decoded

As far as I know the internal format of the ACBLscore game files has never been published. The information presented below is the result of a few days of reverse engineering, most of it spent staring at a hex editor display much like the screenshot below. This documentation is not complete but it is quite detailed and should be a good start for anyone else. I update it from time to time but an exhaustive reverse engineering would be very time consuming. The effort also seemed pointless because ACBLscore+, the long awaited successor to ACBLscore, was due in perhaps a year. But the ACBL discontinued ACBLscore+ development in July 2014 with little explanation, stating that ACBLscore would be tweaked to meet some of the needs that ACBLscore+ was to address. This continuance of ACBLscore makes a documentation of its internal file format more relevant.

Example of a Pair Entry in a Pair Entry Table in an ACBLscore game file. Entry length is 0x192 = 402 bytes (red box). Pair percentage is 0x1505 = 5381 ⇒ 53.81% (purple box). Last name of first player in partnership is "Angione”, seven ASCII characters (green box).

Why even bother at all? The initial motivation was to compute the field strength and display each player’s masterpoint total in ACBLmerge. Both of these features require the player numbers in order to lookup each player’s current masterpoint total. ACBLscore used to print the player numbers on the text and HTML reports but a couple of years ago ACBLscore stopped including them in most cases, perhaps to make the report format more compact. Since the player numbers had to be in the ACBLscore game files, I went looking for them. It was a fairly easy to search the entire game file for strings that looked like player numbers and then find the corresponding player names which were always at a fixed offset from the player numbers. This simplistic method works pretty well though it can lead to incorrect results if two players have the same first and last names in a single game file, a rare event that nonetheless has happened at regionals where many events are contained in a single game file. A better understanding of the internal format would allow each player number to be associated with a section, pair number, and direction (N-S or E-W).

The second reason is to perform statistical analyses on players and partnerships across many sessions, typically in a regular game, say the La Jolla unit game or Thursday evening at the Soledad Club. For the Social Network graphs, it was enough to pull additional pair data using the same technique that ACBLmerge uses. But the Payoff Matrix requires more detail, specifically results from every board. It is possible to do these analyses by parsing the text and HTML reports generated by ACBLscore. However, there are many little quirks involved in the text parsing. And different clubs choose to generate different ACBLscore reports. Worse, I don’t know any way to systematically generate a particular style of report from the ACBLscore games files, i.e. from the command line; it is quite tedious to manually open several hundred game files in ACBLscore and create the desired report for each one. Directly reading the binary game files solves both these problems.

Third, some information in the game files is not readily obtainable elsewhere. For example, an analysis of whether some pairs routinely choke on the last round requires knowing which round each pair played each board. For a Mitchell movement this is straightforward to calculate. For a Howell movement it is necessary to know the exact movement. But when parsing game files directly, the round information is available from game file itself.

Finally, directly parsing the binary format is really fast. For example, my Payoff Matrix Perl program scans through 600 games files in about four seconds on a Windows 7 era laptop. It wouldn’t be intolerably slow to scan the text based reports but performance is always a win. Users don’t like to wait, especially when the user is also the developer.

Acknowledgments

Don Greenwood sent me his GameScan C# code for parsing ACBLscore games files in early January 2014. At the start of March I was finally able to integrate my reverse engineering with his reverse engineering to produce the most complete and accurate documentation to date. In the middle of September he sent me updated code which I have used to further refine the documentation.

Ping Hu unraveled some of the remaining mysteries of the Section Details structure in late September, including the initial seating arrangement.

Data Types, Data Layout, and Notation

It is worth making several general observations which will simplify the rest of the documentation.

• Data is little-endian.
• File pointers are 4-byte integers representing the offset from the start of the file and always point to the start of a block.
• Blocks are variously termed structures or tables where a table is a particular kind of structure (see below). The first two bytes of each block give the block length, not including the two bytes specifying the block length. Not all structures are blocks.
• Most floating point values have a fixed number of digits after the decimal (usually two) and are stored as integers.
• All strings are ASCII, have a fixed maximum size, and are preceded by a single byte indicating the number of characters in the string.
• Some tables have a corresponding index table with entries containing file pointers to the entries in the original table.

Let uint and int designate unsigned and signed integers respectively. Let uint8, uint16, and uint32 designate a 1, 2, and 4 byte unsigned integers respectively where int8, int16, and int32 are the corresponding signed types. Let pointer designate a 4-byte integer file pointer, a uint32 that happens to be a pointer; zero indicates a NULL pointer. Let boolean designate a uint8 meant to be interpreted as true (non-zero) or false (zero).

There can be some ambiguity about the size of a field. Often a zero always seems to appear after what might be reasonably be stored in a uint8, leading to the assumption that it is really a uint16. Similarly, some conceivable uint16 values are always followed by a pair of zeros, leading to the conclusion that the data is interpreted as a uint32 value. This documentation uses the most reasonable interpretation. Please report assumptions that are falsified.

Most floating point values have exactly two digits after the decimal point. For example, the percentage 53.81% is stored as 5381 which has the hexadecimal representation of 0x1505 and hence the byte order 0x05 0x15 since data is stored little-endian. Both matchpoint and masterpoint quantities are handled in the same manner. Let the intfloat(n) designate a fixed decimal value with n digits after the decimal point. I have seen no evidence of standard IEEE-754 floating point values in the ACBLscore game files. Most intfloat values are intfloat(2) data types have been observed but intfloat(4) is seen for the masterpoint M-factor for each strat and intfloat(1) is seen elsewhere. Some intfloat(2) values appear to occupy 2 bytes which others most likely occupy four bytes. Some intfloat values are signed. Eventually a few Pascal real48 values were identified. Some of these seem to duplicate existing fields and were probably added to combat round-off error.

Let string(n) designate a string containing up to n ASCII characters. These fields occupy n+1 bytes because the first byte indicates the actual length of string. The unused part of the string field is almost always zero padded but this is not completely reliable and even if it were, relying on a terminating NUL will fail whenever the string length is as long as longest string allowed. Use the first byte to determine the string length. Let char(n) designate an array of n bytes intended to be interpreted as characters, e.g. ‘A’ without nul termination. In practice only char(1) has been relevant.

Let datetime designate a uint32 which represents a date and time after Jan 1, 1980 decodable as shown in the table below, where date = datetime >> 16 (high bytes) and time = datetime & 0xFFFF (low bytes). Note: this encoding only allows a resolution of 2 seconds, rather than 1 second.

Let struct mean a C style structure where members are laid out at fixed offsets from the start of the structure. Let table mean a structure whose primary purpose is to contain many entries (records) of another structure. Tables usually consist of a table header followed by multiple table entries. Usually the table entries start immediately after the table header and are laid out without gaps. However, it is unsafe to rely on this if the table has a corresponding index table. Use the index table to find the starting location of each entry in the table.

None of the structure or table names described here are official names. This is strictly a reverse engineering effort.

The word “flag” is used to mean that a value should be interpreted as a Boolean. Typically a uint8 or uint16 is interpreted as such.

All relative offsets shown in blue in the charts below are in hexadecimal, even though they are not prefixed with 0x. All other values should be assumed to be decimal values unless prefixed by 0x.

Block Layout

An ACBLscore game file consists of a set of blocks where the first two bytes (int16) give the size of the data in the block. The Master Table and the Pair, Team, Events Details, and Section Details structures as well as the Pair Index, Team Index, and Board Results Index tables are all examples of data which fully occupy a block. Other structures presented below are never allocated separately in their own block but instead are constituents of one or more block allocated objects. For example the Player Structure appears twice in the Pair Structure and six times in the Team Structure. The Masterpoint Award and Ranking structures appear in multiple places.

Free blocks are designated by a negative size and include a pointer to the next free block as shown below such that the free blocks form a linked list. The Master Table includes a pointer to the first free block.

Block allocation in ACBLscore is closer to memory allocation than disk allocation because there is no minimum unit of allocation corresponding to a physical object (i.e. a 4096 byte allocation corresponding to eight old size 512 bytes sectors or one modern 4096 byte sector). Deleted structures are presumably turned into free blocks and if possible coalesced with adjacent free blocks into a larger free block. When a new structure is allocated, the free block list is traversed until a large enough free block is found whereupon it is converted to an allocated block and a smaller free block. If no free blocks are available, the file size is extended to accommodate a new block at the end. Unlike similar block allocated formats such as the JPEG image format, blocks are not necessarily consecutive. Some file bytes do not belong to either an allocated or free block. This means it is necessary to follow pointers to find all the blocks rather than simply traverse the file linearly.

Freed blocks are dirty, i.e. the block is not overwritten with zeros beyond the next block pointer when it is freed. The particular behavior of ACBLscore makes it common for freed blocks to look like a Pair Index Table.

Entity Relationships

The diagram above shows the relationship between the entities documented below. Blue boxes indicate block allocated entities. Black arrows represent pointers. These always point to block allocated entities. Yellow boxes represent entities which are constituents of one or more block allocated entities. Dashed blue arrows originate from entities that contain the constituent entity that the arrow ends on. Numbers along an arrow represent the number of pointers or the number of times the constituent entity is contained in the parent entity.

Some entities are grouped for simplicity in the diagram. In some cases it may make sense to consolidate these entities into a single entity in the documentation. For example, the Individual, Pair, and Team Structures appear to have a common start which is then followed by one, two, or six Player Structures respectively. Don Greenwood’s GameScan program consolidates these as a single Entry Structure (as in tournament entry). The Individual, Pair, and Team Match Table could be consolidated but they are small and the consolidation would likely make the documentation harder to understand. Likewise, the Board2 and Board4 Structures, for pair and individual events respectively, could also be consolidated but only at similar cost.

The inclusion of the Masterpoint Award Structure in both the Individual/Pair/Team Structures and the Player Structure may seem odd. But it really works this way. For team games this makes sense because up to six players are allowed on a team and the individual players can end up with different awards depending on who is playing when. For example, to qualify for the main awards, a player must play at least 50% of the time but could qualify for a match award simply by playing for the duration of a match. It is not clear if the two players in a pair can end up with different masterpoint awards. The individual case is clearly silly, no doubt a side effect of implementing pair and team games first.

Events and Sections

Each ACBLscore game file contains one or more events. Examples of events are Open Pairs, 0-750 Pairs, Swiss Teams, Knockout Teams, etc. Each event contains one or more sections. Sections are usually designated by single or doubled letters, i.e. A-Z or AA-ZZ, though numerical designations such as 11–15 have been seen for Compact Knockouts. In this documentation, a section refers to both the N-S and E-W pairs, i.e. both directions. The use of section in ACBL documentation is inconsistent. Sometimes it has the meaning above; other times it means only one direction, e.g. when an section masterpoint award is computed for the each direction of a lettered section in a Mitchell movement.

Master Table and Section Summary Structure

An ACBLscore game file begins with the Master Table. Therefore in this case only, the relative file offsets are identical to the absolute file offsets.

The ‘AC3’ string seems like a reliable way to test whether a file is an ACBLscore game file, i.e. the bytes of the ‘AC3’ string are good magic bytes. The Master Table length can probably also be used as part of the magic bytes because the limits of 50 event and 100 sections are hard coded into ACBLscore according to the ACBLscore manual.

Section and Event numbering begin at 1. The Event number associated with each section can be used to determine which sections belong to a specific event. But this is not the whole story. Within an event, sections can be partitioned into groups where the sections in each group are scored together. Sections in each group are said to be combined with each other in ACBLscore parlance. See section 3.2.9.2 of the ACBLscore Manual (CFIG2 command). Scored together means the matchpointing is performed across all combined sections. For example if a board is played nine times in both section A and B, the matchpoint top will be 8 if the sections are scored separately and 17 if the sections are combined for scoring. When sections are scored together (combined), all pairs from each direction (N-S or E-W) are ranked for the overall awards. Within a group of combined sections, sections may be grouped again for section award rankings. For example if sections A and B are combined and also ranked together, the section awards will be based on all N-S pairs and all E-W. If they are not ranked together, their will be separate section awards for the N-S pairs in A, N-S pairs in B, the E-W pairs in A, and the E-W pairs in B. Ranking sections together increases the section award for first place but has little impact on the overall masterpoints awarded. Gold Rush pair events are often ranked separately in each section because only the top section rank gets gold points and a little gold for the top pair in each section is deemed more desirable than more gold for the single pair in these events that have become the ACBL equivalent of a diploma mill.

The partitioning of sections for scoring and ranking is handled by what is effectively a pair of forward and backward linked lists where 0 is the null pointer at the end of each list.

For pairs events, the score on each board is scaled up to the matchpoint top (N-1) when the number of results for the board is fewer than N. This can happen when a table does not have time to play the board (and there is no late play), there is a sitout in one section, there are differing numbers of tables in each section such that some boards (usually high numbered one) are not used at all in some section(s), or one or more sections do not complete all scheduled rounds. This scaling can lead to curious looking matchpoint score. For example, consider this ugly situation of a pairs event with a 12½ table section and a 12 table section. Each section played two board rounds. ACBLscore come up with a combined top of 23 on the assumption that a board could be played a maximum of 24 times, 12 times in each section; however the 21½ section which by itself would be played 13 rounds was terminated after 12 rounds to end at the same time as section B. Due to the sitout in Section A, most boards were played 11 times so the results were scaled by 23/22. Boards 25 and 26 were only played in Section A so they got scaled by 23/10. Even this is not quite right because the scaling is not to 0 and N-1 but rather to a slightly smaller interval in an attempt to make a top based on more results a bit more worthy than a top based on fewer boards. This explains the tops and bottoms of 22.98 and 0.02 respectively instead of 23 and 0 for board subject to the nominal scaling by 23/22.

Note: the ACBLscore documentation confusingly uses the term factoring instead of scaling, despite the completely different meaning of the word factoring in algebra. Don Greenwood does not share my aversion to factoring, citing a definition of the term meaning, “to bring to a common base.”

Event Details Structure

Each event has a pointer in the Master Table to an Event Details structure.

Section Details Structure

Each Section Summary Structure has a pointer to a Section Details Structure.

Howell Pair Number Mapping Structure

Suppose you want to find the Pair Structure for pair 7 as listed on an ACBLscore text report. Examine the byte at offset 0x6 (7-1) in the present structure, which will correspond to offset 0xdd + 0x6 = 0xe3 in the Section Details Structure. If it is non-zero, replace the original pair number with the value found, otherwise keep your pair number. Suppose it wasn’t reassigned, the usual case. Examine the value at offsets 0x50 + (7-1) = 0x5c and 0x5d in the present table to determine the initial table and direction for the pair. Suppose the values are 3 and 2 (E-W) respectively. Use the E-W Pair Index Table pointer located at offset 0x4 + 4*(2-1) = 0x8 in the Section Details Structure which has the value p. The initial table number for the pair corresponds to an entry in the Pair Index Table. Skip over the 0x14 byte header of the Pair Index Table and the first 4 bytes of the 8 byte Pair Index Entry, to find the pointer to the Pair Structure at p + 0x14 + 0x4 + (3-1) * 8 = p + 0x28.

Mitchell Pair Number Mapping Structure

Note: a web movement is treated as a Mitchell movement as far as player seating and movement is concerned. It is only the movement of the boards that is slightly affected by the web movement.

Masterpoint Award Structure

The Masterpoint Award Structure occurs in the Player, Pair, and Team structures.

If only one pigmentation type, e.g. Black, is awarded, the result will always be stored in the first position. If only two pigmentation types are awarded, the results will always be stored in the first two positions.

Mixed pigmentation occurs in several ways. It can occur in a single session when a pair ranks high enough to receive an award in a superior pigmentation (e.g. from an overall ranking) while simultaneously ranking high enough in a lower strat or in a section to received a higher award in an inferior pigmentation. In this case the pair receives the highest masterpoint award but only the part of it that would have been awarded in the superior pigmentation is given in the superior pigmentation. The most common scenario is a mix of gold and red points at a regional. Or a pair can received gold in one session and red in another of the same event. Also some events, e.g. low bracket Knockouts, pay out a mix of pigments.

For handicapped events that are ranked both with and without the handicap, the award for the ranking with the handicap is stored in the first element and the award for the ranking without the handicap is stored in the second element. If a pair only receives an award without the handicap, the masterpoint award for the first position will be zero. For handicapped events, the masterpoint pigmentation type will be the same in each element unlike the normal case above where the pigmentation type is different for each element. If a pair receives an award both with and without the handicap, their total award is the sum of the two masterpoint awards as indicated in these fields. For example, consider a pair that third after the handicap is applied and ranks 1st without the handicap, where the awards are 0.18 MP and 0.53 MP. The pair receives the better of these awards, 0.53 MP. However internally, the awards are stored as 0.18 MP and 0.35 MP respectively, as though the award without the handicap were an inferior pigmentation even though the pigmentation is the same.

Ranking Structure

The Rank Structure occurs in the Individual, Pair and Team structures. This structure stores the section and overall ranking, including ties, for a pair or team in a single strat.

Player Structure

The Player Structure occurs once in the twice in the Individual Structure, Pair Structure and six times in the Team Structure. In team events, team members can end up with different masterpoints, for example when a team member is added in a later round of a Knockout event. I am not sure what distinguishes the two fields that seem masterpoint related or whether they are relevant to pairs events.

The Sectional Master and all higher ranks, also have pigmented point requirements. Grand Life Master requires a national title.

The national board of directors added the Ruby (1500 MP) and Sapphire (3500 MP) ranks at the Fall 2015 NABC meeting (Denver) and they went into affect at the start of 2016. Previously the rank letters ran from A-N. The change in ACBL game files probably took affect only when clubs updated their rosters.

Pair Structure

If there is a phantom pair, the phantom pair will have a Pair Structure associated with it but nothing will be filled in.

Pair Index Table

The Pair Index Table gives the starting addresses of each Pair Structure. Pair Structures are usually laid out consecutively without gaps but this should not be assumed. Use the Pair Index Table to find each entry. For sections belonging to pairs events, the Section Details Structure contains two pointers at offsets 0x0004 and 0x0008 to Pair Index Tables.

A pairs sections usually (always?) has two Pair Index Tables with Pair Index ID of 1 or 2. These index tables are for the pairs sitting N-S and E-W at the first round. This means for a Mitchell movement, they are for the N-S and E-W pairs respectively. But for a Howell movement it is an arbitrary division of the pairs based on the movement.

Board2 Result Table

A Board2 Result Table stores all the results on a board in a single section of a pair event. For individual events, see the Board4 Result Table.

The internal storage of raw scores is similar to the ACBLscore user interface. The superfluous final zero is dropped in both during data entry and in the stored value.

ACBLscore has to track separate raw scores and matchpoints for each direction to support director rulings where each side is assigned a different results, possibly based on different assumed contracts, and/or is assigned and Ave-, Ave, or Ave+ score.

ACBLscore seems to have a small bug in that the pair numbers shown for board results are the pair numbers before accounting for any pair number reassignments (though the names are correct). This issue does not affect the score or award for any pairs.

Board4 Result Table

A Board4 Result Table stores all the results on a board in a single section of an individual event. For pair events, see the Board2 Result Table.

ACBLscore has to track separate raw scores and matchpoints for each player to support director rulings where each player is assigned a different results, possibly based on different assumed contracts, and/or is assigned and Ave-, Ave, or Ave+ score.

Note: ACBLscore reports do not show the South and West individual numbers even though they are tracked internally. Support for individual games was probably added as an extension of the support for pair games.

Board Results Index Table

The Board Result Index Table gives the starting addresses of each Board2 Result Table (pairs) or Board4 Result Table (individual events). Board Result Tables are usually laid out consecutively without gaps but this should not be assumed. Use the Board Result Index Table to find each entry. The Section Summary Structure has a pointer to the Board Result Index Table at offset 0x0008.

Board numbers are usually 1-N in the Index Entries but this can be altered by the EDMOV command, option “1. Display or reassign board sequencing”. See section 3.2.14 of the ACBLscore Manual.

Pair Match Table

The Pair Match Table indicates which pairs are seated at each direction at each round at each table. Use the information about which pairs are seated at each table at round one to determine the pair number for each Pair Structure based on the table number given at offset 0x0002 of the Pair Structure.

Note: The Pair Match Table was formerly called the Movement Structure but a more careful examination of how teams and individual events were stored suggested renaming the table for consistency.

The Pair Match Table allocates 75 bytes per table for movement information, enough for 75 / 3 = 25 rounds. This is enough rounds for one board per round. In practice it is much more common to have two or three boards per round. Unused Pair Match Structure array elements contain zeros in all fields.

Individual Structure

The Individual Structure is essentially the same as the Pair Structure but includes only one player. It is used for rarely played individual events where each individual is matched with a new partner on each round.

Individual Match Table

The Individual Match Table indicates which individuals are seated at each direction at each round at each table for individual events.

The Individual Match Table allocates 180 bytes per table for movement information, enough for 180 / 5 = 36 rounds. This is enough rounds for one board per round. Unused Individual Match Structure array elements contain zeros in all fields.

Team Structure

A team may have up to six members. Team members receive masterpoints if they play at least half the time.

Team Index Table

The Team Index Table gives the starting addresses of each Team Structure. Team Structures are usually laid out consecutively without gaps but this should not be assumed. Use the Team Index Table to find each entry.

Team Match Table

A Team Match Table has all the results for the matches that a single team plays, i.e. the Team Match Entries, and some additional information about the team, including the stationary table requirements for each direction of the team and to what extent those seating requirements can be violated.

Round Robin Structure

Victory Point Structure

This scheme allows for fractions of a Victory Point though this never seems to occur in practice.

Memo Structure

Memos can apply to all events, individual events, or individual sections. Memos are printed on the appropriate recap sheets.

Note Structure

The Memo and Note structures are nearly identical, differing only in the maximum length of each line. But their purpose is different. Memos are information for players and as such are printed on ACBLscore reports. Notes are information for the Director-in-Charge (DIC), other directors, and ACBL headquarters. Notes pop up on screen when the game file is loaded in ACBLscore but are not printed on ACBLscore reports.