The PCGen LST format: Part 1 – Really Long Lines

Category: Games

This information is current as of PCGen v6.05.02, at 2015-06-23.

PCGen is a tool for building characters for tabletop role-playing games, i.e. Dungeons and Dragons 3.5ed or Pathfinder. PCGen comes with all the Open Gaming License material for Dungeons and Dragons 3.5 edition (3.5ed) pre-coded and ready to go, which is very handy.

I wanted to extend the base 3.5ed content with some home-brew material. This is done using the PCGen .LST file format (a collection of tab-separated value files.) Therefore, I am slogging through the existing 3.5ed and Pathfinder .LST files figuring out how they work, so that I can figure out how to write my own.

This begins with figuring out some of the common data entry techniques and patterns used in .LST files. The first of these is how we manage very long records.

Really long lines

The .LST file format has no concept of nested records. One line is one record, and everything to do with that record must be on that one line.

This leads to lines that are thousands of characters long, as in this 1,051-column example from the Pathfinder Advanced Races Guide data file arg_abilities_race.lst:

Adaptable Luck  KEY:Halfling ~ Adaptable Luck   CATEGORY:Special Ability    
            PREMULT:1,[PREABILITY:1,CATEGORY=Special Ability,Halfling ~ 
Adaptable Luck],[!PREABILITY:1,CATEGORY=Special 
Ability,TYPE.HalflingLuck]                      DEFINE:AdaptableLuckTimes|0 
DEFINE:AdaptableLuckBonus|0 DESC:Some halflings have greater control over 
their innate luck. This ability gives them more options for how they can 
apply their good fortune from day to day, but also narrows its scope. %3 
times per day, a halfling can gain a +%1 luck bonus on an ability check, 
attack roll, saving throw, or skill check. If halflings choose to use the 
ability before they make the roll or check, they gain the full +%1 bonus; if 
they choose to do so afterward, they only gain a +%2 

BONUS:VAR|AdaptableLuckBonus|2                  COST:0  SOURCEPAGE:p.62 
ASPECT:CheckCount|%1|AdaptableLuckTimes ASPECT:CheckType|Uses per day

Workaround for really long lines

1,000 character long lines are challenging to deal with in a text editor. Therefore, it is handy to have a way of breaking up a record over multiple lines.

There is a feature called a .MOD record which modifies other records by appending data to an existing record. (It can also overwrite things, which is why it’s called modify and not append.)

This can be abused to rewrite things of the form:

MyAbility  TAG1:value1  TAG2:value2  TAG3:value3 ... [lots of tags] ... TAG99:value99

In the form (for an ability file):

MyAbility CATEGORY:Internal
CATEGORY:Internal|MyAbility.MOD TAG2:value2
CATEGORY:Internal|MyAbility.MOD TAG2:value2
CATEGORY:Internal|MyAbility.MOD TAG3:value3
CATEGORY:Internal|MyAbility.MOD TAG99:value99

This works by defining the “empty” record MyAbility which contains nothing, then repeatedly appending tags to it. This is very verbose, but at least the tags are now broken out onto separate lines.

Surprising syntax

The CATEGORY:InternalRecord|MyAbility.MOD syntax is extremely surprising at first.

All other records are required to have an identifier followed by one or more tags:

# Valid record named MyAbility
MyAbility  CATEGORY:Internal  TAG1:value1 ...

# Record doesn't start with a name. Invalid.      
TAG2:value2  TAG3: value3 ...

# Is this valid? It doesn't look like it starts with a record name...
CATEGORY:Internal|MyAbility.MOD  TAG4:value4 ...

CATEGORY:Internal|MyAbility.MOD looks like it violates this rule, because CATEGORY:... looks like a tag (TAG:...), and you aren’t supposed to start lines with a tag.

In this case, however, the entire string CATEGORY:Internal|MyAbility is actually the name of a record, i.e. the existing record MyAbility in category Internal, so the line does begin with a record name (of sorts.)

A live example

Here’s an example from the Pathfinder data file, cr_abilities_race.lst, of how this is done in practice.

Race Traits ~ Dwarf                 CATEGORY:Internal

CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Defensive Training|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfDefensiveTraining
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Greed|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfGreed
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Hatred|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfHatred
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Hardy|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfHardy
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Stability|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfStability
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Stonecunning|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfStonecunning
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Steady|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfSteady
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Weapon Familiarity|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfWeaponFamiliarity
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Darkvision|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfRacialVision
CATEGORY=Internal|Race Traits ~ Dwarf.MOD       ABILITY:Dwarf Racial Trait|AUTOMATIC|Dwarf ~ Language|!PREABILITY:1,CATEGORY=Special Ability,TYPE.DwarfLanguage

In this case, the Race Traits ~ Dwarf is defined as an empty record, with a CATEGORY tag and nothing else. The subsequent lines repeatedly modify the Race Traits ~ Dwarf record by adding ABILITY tags to it.

Note that the name of the ability Race Traits ~ Dwarf is not sufficient to identify the record which is to be MODified - both the ability name Race Traits ~ Dwarf and the ability’s category Internal must be specified: CATEGORY=Internal|Race Traits ~ Dwarf.MOD.

The documentation for the .MOD syntax calls this out:

CATEGORY=Mutation|Weak Immune System.MOD

Modifies an ability of the category "Mutation", which is called Weak Immune System. Another ability of the same name, which belongs to another category, would not be affected.

Next article in series: Exploring the PCGen file format - Part 2: How abilities are mapped onto races.

From Dave (2016-01-13) we have:

Worth noting, you can specify (in preferences) that long lines can be broken into ‘multi lines’ – just replace multiple tabs with a newline and single tab, that can be much easier to work with. I use vim and that has a macro to toggle back and forth between these variant formats.