Convert the SQLite SRD database to N3 (Specification)
Contents
- class and class_table, which define the core classes. (class_table is used for spells, saves, and base attack bonus tables)
- domain which defines the 36 spell domains, and what spells are in them.
- equipment, anything you can buy or carry that isn't magical, how much it weighs and costs and anything else it does. (tables from the PHB.)
- feat, the character feats; description, dependencies, what benefit they provide.
- item, the magical stuff (tables from the DMG)
- monster, the monsters; hit dice, special abilities, initiatives, all that jazz.
- power, psionic powers
- skill, the skills; stat used, amount it affects, benefits of using it.
- spell, the spells; components, casting time, cost to research, school.
We need a command-line script to convert these tables to N3. The database was originally Andargor's MySQL database, and was converted by CoryDodt to SQLite. The database never contained any relations. Thus, the connections between these tables which should be easy will require us to match things by name, and check for exceptions to get a list of places where things were just spelled wrong.
Implementation
In general, read the database tables and write the RDF/N3 using formatted text; might be no need for an object mapper here if this is write-only.
Let's set some guidelines as a precursor to documenting the N3 file structure.
Use characteristic.n3 and property.n3. The column names should be defined there so someone can create a whole new set of skills without using skill.n3 as a basis (and thus acquiring all of the predefined skills, which requires him to make a blacklist, when he only wants the schema).
skill.n3 should contain only the skills and subskills themselves from the SRD. feat.n3 should contain only SRD feats. And so on. See item 1.
Filenames should be singular, e.g. property.n3 and family.n3
Use RDF/XML to store rich text columns (or maybe all longish text columns).
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns='http://www.w3.org/1999/xhtml'> <rdf:Description rdf:about="http://thesoftworld.com/2007/skill.n3#jump"> <c:fullText xmlns:c="http://thesoftworld.com/2007/characteristic.n3#"> <p>The DC and the distance you can cover vary according to the type of jump you are attempting (see below).</p> <p>...</p> <table width="100%" border="1" cellpadding="2" cellspacing="2" frame="VOID" rules="ROWS"> ... </table> ... </c:fullText> </rdf:Description> </rdf:RDF>
This doesn't accomplish what I was hoping, which is to put (X)HTML into an editable environment instead of an opaque string. HTML editors can't edit this, and it's not even really readable the way it comes out of etree because there are namespaces on all the HTML nodes. What other options do we have here?
Maybe just a plain XHTML document with special id attributes on the top div of each part?
Jerub had a good idea (I'll give him credit even though he modestly said it came from someone else); for the boolean columns such as psionic, trained and untrained, represent them as classes the node possesses. For example:
:jump a c:Skill; a c:HasArmorCheckPenalty; p:keyStat c:str; . :knowledge a c:Skill; a c:TrainedRequired; p:keyStat c:int; p:subType :arcana, :architectureAndEngineering, ...; .Stats that can be computed trivially should be, rather than storing them in the data. See related Open Questions below.
Skill - Done! bin/ptconvert skills > skill.n3
Feat - Done! bin/ptconvert feats > feat.n3
- There is a feats_preamble.n3 but it's empty. I didn't attempt to convert prerequisites or choices; rdflib suffers from issues making these hard to implement, to wit:
- I want to use formulas for choices. The owl:allValuesFrom is a great thing to use in a choice formula, but a bug in rdflib means prefixes that are ONLY used inside formulas will not be serialized. Thus rdflib is producing an unparseable document.
- Also, a!b!c references are needed, but rdflib creates a bnode when using that kind of reference. When this happens inside a formula, the bnode never gets defined as a subject anywhere, so information is thrown away.
Monster
Use goonmill.history.History to get these stats out.
Class/Class_table
Equipment/Item
Power
Spell
Open Questions
- Do tables need to be converted completely, or can we start with just the names of skills and feats and make it possible to reload at a later point, read more columns from the database, and rewrite?
This question really needs to be answered before we go any further because it determines whether or not we need to implement a round-trip converter for every table. I'm in favor of doing a complete conversion of every table so no round-trip is necessary. - CoryDodt
- What's the cutoff for stats that can be computed trivially and stats that shouldn't be? Example: "Flat-Footed AC" is regular AC sans Dex bonus. "Touch AC" is 10+Dex bonus; both of these are trivial to compute, so they probably should be computed rather than included in the data. But what about the AC score itself? It can be computed as (10+Dex Bonus+Armor+Shield+Size penalty+natural armor/racial bonus). Should it then be computed, or should the rest of the attributes be computed instead?
Formulae? One thought: Formulae should just be applied in-place, as it were, with an inference engine. Therefore, published data files would be the results of that inference (can be done by cwm.py --think). But if we make the inference engine a requirement of the consumer APIs of Playtools, then we can use formulae to encode the rules themselves. Formulae that describe the behavior during gameplay events cannot be precomputed into our data. For example, in family.n3 there are formulae that describe how a demon changes when it is outside its plane of existence. I don't think it makes sense to precompute that data; rather the formula that describes it should be interpreted by the engine simulating the instance of a demon.