In my last Faxanadu update I had already hinted at the fact that you need to understand how to leave screens if you want to understand which tiles are used for a certain screen. This is because the level design of Faxanadu is an inductive process. The tiles used in a screen depend on the tiles used in the screen before. The same is even more true for the colors used to paint a screen. Before going on with explaining how a screen is drawn it's therefore necessary to know about the different ways of entering and leaving a screen. This update deals with the doors, the next update will deal with scrolling.
One might assume that there are just two ways to leave a screen, through a door or on one of the four edges of the screen. Nothing is that simple in Faxanadu though. There are (at least) five different ways to leave a screen, three of them are about leaving screens through a door, the other two are for leaving a screen on one of its edges. That's the five ways I've uncovered so far at least and they're sufficient to draw all screens correctly in my level editor.
Let's start with the doors. There are three different types of doors although they all look the same. The first one is used to move between different parts of one level, the second one is used to move between different levels and the last one is used for entering buildings. Strictly speaking there are actually four different types of doors because the game distinguishes between two different types of doors for moving between levels. One type is for moving one level forward and the other type is for moving one level back.
Faxanadu stores all information about doors in two tables for each of the eight level data chunks. The first table is the door location table. It contains information about the position of doors. The other table is the door destination table, it provides information about what happens when the player enters a door. How these two tables are located can be found in Vagla's document about the Faxanadu internals.
Vagla already does a good job explaining the format of the door location table. His information about this table is absolutely correct so I'm not going to reiterate it. The other table is the door destination table. The individual entries of that table are also four bytes long but the exact meaning of those four bytes depends on the type of door in that entry.
To distinguish between the different types of doors it's necessary to check the third byte of a door entry in the door location table. If it's lower than 0x20 it's an index into the door destination table and the door is used to move between different screens within the same level. If it's 0xFE the door is used to move back one level, if it's 0xFF the door is used to move to the next level. 0xFE and 0xFF can not be used as indices into the door destination table. If the value is between 0x20 (including) and 0xFD (including) the door is used to enter a building. The value is also used as an index into the door destination table again.
As I've said, nothing is simple in Faxanadu. There's an exception. In level 2 (the towns) the door values behave differently. Before checking the value to find out the door type 0x20 is subtracted from the value.
Let's look at the door destination table now which is relevant for door destination values less than 0xFE. I've already mentioned that the individual entries in this table are also four bytes long. The meaning of the first two bytes changes depending on the door type (enter building or move between screens). The third byte specifies what is required to enter the door (key, ring, nothing; see table at the bottom), the fourth byte is unused and always 0.
When entering a building the first byte specifies the NPCs found inside, the second byte specifies the building that door leads to (the screen ID). When moving within one level the first byte specifies the screen ID of the screen the door leads to and the second byte specifies the palette ID of that room.
Things are a bit different for doors of value 0xFE and 0xFF. There are some tables in the Faxanadu ROM file that are used when moving between levels. The first one (located at offset 0x3E5F7) contains the levels you end up in when moving between levels. The current level is used as an index into that table, the value read from the table is the level a door from the index level leads to. There are two values for each level, the first one is for going back one level, the other one is for moving forward. Here's the original table.
Combined with the information from the first table the screen a door leads to is well-defined. If you're in level 2 and go back you'll end up in level 1/screen 28. If you go forward you'll come to level 3/screen 0.
It needs to be mentioned that the indexes into these two tables don't follow the regular numbering of levels. Here are the meanings of the valid values 0 to 5: 0 = First town; 1 = First world; 2 = Fog world; 3 = Tree world; 4 = Last level; 5 = Labyrinth before the final boss.
The code where this type of door is handled can be found in the last ROM bank between the offsets 0xE5B2 and 0xE5D7. It uses a third table (offset 0x3E5EB) of the same structure which contain the door requirements of each door (nothing, key, ring).
Here's the original door requirements table (the values can be found at the very bottom of this update):
$E5b2-> 4a: LSR A
$E5b3-> ad 35 04: LDA $0435
$E5b6-> 2a: ROL A ; A is an index into the tables
$E5b7-> 48: PHA
$E5b8-> a8: TAY
$E5b9-> b9 db e5: LDA $e5db,Y ; Load the door requirements (Key, ...)
$E5bc-> 8d 2b 04: STA $042b
$E5bf-> 20 2f eb: JSR $eb2f ; Check if the player has a valid key.
$E5c2-> 68: PLA
$E5c3-> a8: TAY
$E5c4-> ad 2b 04: LDA $042b
$E5c7-> d0 11: BNE $E5da ; No valid key
$E5c9-> b9 e7 e5: LDA $e5e7,Y ; Load the section ID of the new screen
$E5cc-> 8d 35 04: STA $0435
$E5cf-> b9 f3 e5: LDA $e5f3,Y ; Load the screen ID of the new screen
$E5d2-> 85 64: STA $64
$E5d4-> 20 2f da: JSR $da2f
$E5d7-> 4c dc da: JMP $dadc
There are two other parts in the code that are very important for handling doors, both are located in the last ROM bank. The first one starts at offset 0xE815 and ends at 0xE85F. This part is responsible for looking up information about a door in the door location table and door destination when the user pressed the up button while standing in front of a door.
At first there's a loop that attempts to find the door in the door location table.
$E815-> a0 00: LDY #$00
$E817-> b1 02: LDA ($02),Y ; Load screen ID of the next door from the
; door location table.
$E819-> c9 ff: CMP #$ff
$E81b-> f0 45: BEQ $E862
$E81d-> c5 63: CMP $63 ; Compare it with the screen ID of the door to enter
$E81f-> d0 31: BNE $E852 ; If they're not equal check the next door (code omitted).
$E821-> c8: INY
$E822-> b1 02: LDA ($02),Y ; Load position of the next door from the
; door location table.
$E824-> c5 6a: CMP $6a ; Compare it with the position of the door to enter
$E826-> d0 2a: BNE $E852 ; If they're not equal check the next door.
Now information is read from the door location table and there's preparation for accessing the door destination table.
$E828-> c8: INY
$E829-> b1 02: LDA ($02),Y ; Index into door destination table.
$E82b-> 85 6b: STA $6b
$E82d-> c8: INY
$E82e-> b1 02: LDA ($02),Y ; Door position in the new screen.
$E830-> 85 6c: STA $6c
$E832-> a5 6b: LDA $6b
$E834-> a4 24: LDY $24
$E836-> c0 03: CPY #$03 ; Compare if the door is in a town
$E838-> d0 03: BNE $E83d
$E83a-> 38: SEC
$E83b-> e9 20: SBC #$20 ; If it is, subtract 0x20
$E83d-> 0a: ASL A ; Multiply index into destination table
$E83e-> 0a: ASL A ; by four (= size of one entry).
The last part reads the door destination table.
$E83f-> a8: TAY
$E840-> b1 8f: LDA ($8f),Y ; Read the first byte (Meaning depends on door)
$E842-> 85 64: STA $64
$E844-> c8: INY
$E845-> b1 8f: LDA ($8f),Y ; Read the second byte (Meaning depends on door)
$E847-> 85 65: STA $65
$E849-> c8: INY
$E84a-> b1 8f: LDA ($8f),Y ; Read the third byte (key requirement)
$E84c-> 8d 2b 04: STA $042b
$E84f-> 4c 66 e8: JMP $e866 ; Everything was read succesfully
The other major relevant part of the code is the code that handles the information read from the door destination table. It begins at offset 0xE526 and ends at offset 0xE568.
$E526-> a5 19: LDA $19
$E528-> 29 08: AND #$08
$E52a-> f0 3c: BEQ $E568 ; No door
$E52c-> 20 c5 e7: JSR $e7c5
$E52f-> a5 b7: LDA $b7
$E531-> f0 35: BEQ $E568 ; No door
$E533-> a5 6b: LDA $6b
$E535-> c9 fe: CMP #$fe
$E537-> b0 79: BCS $E5b2 ; If this jump is taken the door
; leads to another level.
$E539-> 20 2f eb: JSR $eb2f
$E53c-> ad 2b 04: LDA $042b
$E53f-> d0 27: BNE $E568
$E541-> a5 6b: LDA $6b
$E543-> c9 20: CMP #$20
$E545-> b0 30: BCS $E577 ; If this jump is taken the door
; leads into a building.
$E547-> 20 2f da: JSR $da2f
$E54a-> a2 06: LDX #$06 ; If this part is executed the door
... ; is used to move within one level.
I'm going to close this update now with the table that shows the possible values that can be used for the third byte of the door destination table, the key requirements.
0 No key required.
1 Key A required
2 Key K required
3 Key Q required
4 Key J required
5 Key Jo required
6 Ring of Elf required
7 Ring of Dworf required
8 Demon's Ring required