Home › Database access › Gameplay XML data format

Engine executable. Settings and tools of the game program (CivilizationVI.exe), usable without a mod. Mostly undocumented debug and configuration functionality; descriptions come from analysis of the executable and have not all been tried in the running game.

Gameplay XML data format

The XML files that fill the game databases (the GameData/GameInfo files of the base game, DLCs and mods' UpdateDatabase actions) are read by one loader in the exe, Database::XMLSerializer. This page lists the elements and attributes the loader recognises and the SQL each one produces. Verified: names and behaviour read from the decompiled loader functions (addresses in the tables); the shipped-file counts are from 1562 data XML files of the installed game (names only). The loader itself is the same for mods; which tables exist is defined by the game (see the database statements and the GameCore reference).

What the counts show. The shipped files use Replace (382553) and Row (115271) for almost everything, plus Update (1693), Delete (10753) and InsertOrIgnore (27). They define only 8 tables with Table elements (the three files Color_Tables.xml, Diplomacy_Tables.xml and Leader_Tables.xml in the Assets/Gameplay/Data/Schema/ folder of the base game; the main schema is SQL, 01_GameplaySchema.sql and 02_AddTriggers.sql in the same folder); Index, DeleteMissingReferences, InsertOrAbort and the column attributes autoincrement and reference appear in no shipped file, so they are known only from the loader code. Column types used in shipped table definitions: text (31), float (4), integer (1).

Structure

An illustrative file (invented names) showing each construct:

<GameData>
  <Table name="MyThings">
    <Column name="ThingType" type="text" primarykey="true"/>
    <Column name="Cost" type="integer" notnull="true" default="0"/>
  </Table>
  <MyThings>
    <Row ThingType="THING_A" Cost="10"/>
    <Replace ThingType="THING_B" Cost="20"/>
    <Update><Where ThingType="THING_A"/><Set Cost="15"/></Update>
    <Delete ThingType="THING_B"/>
  </MyThings>
</GameData>

Values are given as attributes of the row element or as child elements named after the column (both forms occur in the shipped files); a column named twice is an error. For tables in the loader's package list (it keeps a list of table names) the loader adds a PackageID column with the id of the package being loaded to the insert (PostColumnInsert, 0x79d7b0); which tables are in that list was not read out.

Root elements

Name Meaning SQL Function Shipped uses
GameData Root element of a gameplay data file. The three root names are accepted equally. XMLSerializer (root dispatch) (0x79d3a0) 544
GameInfo Root element, same as GameData (older style). XMLSerializer (root dispatch) (0x79d3a0) 1012
Database Root element, same as GameData. XMLSerializer (root dispatch) (0x79d3a0) 6

Children of the root

Name Meaning SQL Function Shipped uses
Table Defines a table; its children must all be Column elements (at least one). The table name is the name attribute. create table <name> (<columns>, PRIMARY KEY (...), foreign key (...) references ...) BuildTable (0x799e80) 8 in 3 files
Index Creates an index; needs name and table attributes and at least one Column child with a name attribute. CREATE INDEX IF NOT EXISTS '<name>' on <table> (<columns>) IndexTable (0x79bdc0) 0 (not used by any shipped file)
DeleteMissingReferences Cleans up rows whose foreign keys point at rows that no longer exist; the table and column attributes pick what to clean (the code reads the foreign key list of each table). DELETE FROM <table> WHERE <column> NOT IN (SELECT <column> FROM <referenced table>) DeleteMissingReferences (0x79ab90) 0 (not used by any shipped file)
(any table name) Any other child of the root names a table; its children are row operations (Row, Replace, InsertOrIgnore, InsertOrAbort, Update, Delete). Anything else is an error ("Row or Delete expected"). UpdateTable (0x79e890) 7118 in 1456 files

Row operations (children of a table element)

Name Meaning SQL Function Shipped uses
Row Inserts a row. The attributes and child elements of the element are the column values. insert into <table> (<columns>) values (...) InsertRow (0x79c380) 115271 in 1256 files
Replace Inserts a row, replacing an existing row with the same key. insert or replace into <table> ... InsertRow (0x79c380) 382553 in 126 files
InsertOrIgnore Inserts a row unless a row with the same key exists (the existing row is kept). insert or ignore into <table> ... InsertRow (0x79c380) 27 in 6 files
InsertOrAbort Inserts a row and fails on a key conflict. insert or abort into <table> ... InsertRow (0x79c380) 0 (not used by any shipped file)
Update Changes rows. Needs exactly one Set element (the new values) and may have one Where element (the condition; shipped files contain Update elements without Where, which then change every row). Both take attributes or child elements, conditions are joined with AND. UPDATE <table> SET <col> = ? ... WHERE <col> = ? AND ... UpdateRow (0x79d930) 1693 in 195 files
Delete Deletes rows; the attributes are conditions joined with AND (equality only, a column named twice is an error). DELETE FROM <table> WHERE <col> = ? AND ... DeleteRow (0x79af50) 10753 in 144 files

Update parts and key columns

Name Meaning SQL Function Shipped uses
Where Condition of an Update (attributes or child elements, at least one, at most one Where per Update). UpdateRow (0x79d930) 1656
Set New values of an Update (attributes or child elements, at least one, exactly one Set per Update). UpdateRow (0x79d930) 1693
Column A column of a Table (or a key column of an Index). BuildTable (0x799e80) 36

Column attributes (inside a Table definition)

Name Meaning SQL Function Shipped uses
name Column name (required). BuildTable (0x799e80) 36
type SQL type name written as is into the create statement. Default when missing: integer. BuildTable (0x799e80) 36
unique Boolean: adds "unique". unique BuildTable (0x799e80) 24
notnull Boolean: adds "not null". not null BuildTable (0x799e80) 22
primarykey Boolean: the column joins the PRIMARY KEY clause (several columns make a composite key). PRIMARY KEY (...) BuildTable (0x799e80) 15
autoincrement Boolean: with the key, writes "primary key autoincrement". primary key autoincrement BuildTable (0x799e80) 0 (not used by any shipped file)
default Default value of the column (written as a quoted literal, or NULL when empty). default <value> BuildTable (0x799e80) 2
reference Foreign key: the attribute value is written after "references" in a foreign key clause (no shipped file uses it, so the accepted value form, table or table and column, is not confirmed). foreign key (<col>) references <table> BuildTable (0x799e80) 0 (not used by any shipped file)

Error messages of the loader

Messages the loader reports (they are also on the messages page). Every one is prefixed Database::XMLSerializer (<file>):, and a failed table update adds "In XMLSerializer while updating table T from file F." or "... building table ...".

Message Function
'Column' element inside of 'Index' requires 'name' attribute. IndexTable
'Column' expected, got '%.*s'. BuildTable
'Index' requires at least 1 child 'Column' element. IndexTable
'Row' or 'Delete' expected, got '%.*s'. UpdateTable
'column' attribute expected. DeleteMissingReferences
'name' is a required attribute of 'Index'. IndexTable
'table' attribute expected. DeleteMissingReferences
'table' is a required attribute of 'Index'. IndexTable
<Set> Needs at least one attribute or child element. UpdateRow
<Update> element requires a child <Set> element. UpdateRow
A database error occurred while creating an index. IndexTable
Cannot parse file. Error at Line = %d, col = %d LoadFromMemory
Could not determine data type for attribute. (Value: %.*s) UpdateRow
Duplicate <Set> elements are not allowed. UpdateRow
Duplicate <Where> elements are not allowed. UpdateRow
Duplicate column names detected in <Delete>. Name: %.s, Value: %.s DeleteRow
Duplicate column names detected in <Row>. Name: %.s, Value: %.s InsertRow
Empty InsertRow
Input XML does not contain database entry tags. GameData, GameInfo or Database LoadFromMemory
Table contains no 'Column' elements. BuildTable
There is a size mismatch between columns (%i) and values (%i) DeleteRow
There was an error executing the delete statement! See Database.log for details. DeleteRow
There was an error executing the update statement! See Database.log for details. UpdateRow