PokeHacking

Join our Discord community

How to expand the usable ARM9 memory data in IV Generation games: a synthetic overlay approach

Mikelan98, Nomura

Target games:


Introduction

Unlike every other stuff in the ROM images -like graphical, binary or sound resources found in the filesystem- the code files cannot be directly expanded, for they have an absolute location and size in the RAM memory. These files include arm9.bin, arm7.bin and the overlay files. Code files contain all the assembly code the game must execute for working properly, along with important binary data (map headers, starter Pokémon or the type effectiveness table, for instance). This means that, if we want to add new routines or expand the binary data, we must find unused memory regions in the code files, which are very infrequent and very short.

That's why finding a way to load code data in the RAM memory (and keep it loaded while the game is running) would solve some obstacles related with the arm9.bin/overlay files expansion issue. This strategy resembles the performance of the overlay files in the RAM, which are selectively loaded when needed, but in our case we would need a "expansion" file always loaded. For that purpose, we will have to find RAM memory regions that are never used by the game, so we can place new data there without the game overwritting it.


Research

Firstly, with the purpose of finding a region of free RAM memory where allocate our data, we used an unofficial DeSmuME release created for tracking unused memory addresses with ADAS (Pokémon Diamond, Spanish ROM), CPUE (Pokémon Platinum, English ROM) and IPKE (Pokémon HeartGold, English ROM). We set 0x10000 (65536) bytes as minium memory region size, and 0x02000000 and 0x02FFFFFF as search range (they are the main memory limits usable by the RAM, as GBATEK states).


Unused memory report for IPKE (Pokéon HeartGold).

Similar reports were obtained in the three games, so we took the four bigger memory regions and rounded them: 0x023C8000, 0x027C8000, 0x02BC8000 and 0x02FC8000, ensuring at least 96 KB of free RAM memory in each one for all the IV Generation games. However, we found out that the data written in these addresses are cloned between them, so everything written in one address will be quadruplicated in the four addresses. That means that we cannot use these four memory regions, but only one (as the others are simply "mirrors"). We chose 0x023C8000 for our purpose, because the other memory regions may be undefined in other emulators or devices.

The next step is to find a way for loading data in the RAM memory permanently. Instead of trying to create a new overlay file in the ROM filesystem (complicated and possibly buggy), we used a subroutine at 0x020064F0 (ADAE, APAE), 0x02006AA4 (CPUE) or 0x02007508 (IPKE, IPGE). This subroutine is used by the game for loading a specific file from a NARC to a specific address in the RAM memory, allowing us to load stable code and binary data in 0x023C8000. This file would be working as a "synthetic overlay", for it is a file present in the ROM filesystem that can contain both routines and binary data and write it to the main memory.

This subroutine (hereinafter named as LoadFromNARC() subroutine) takes three arguments: the first one for the RAM address where data will be loaded, the second for the NARC ID where the file that will be loaded is, and the last one for the file ID within the NARC. These three arguments are taken from R0, R1 and R2 respectively at the time the subroutine is executed.

02007508PUSH
0200750ASUB
0200750CMOVSimm
0200750ELSLS
02007510LDR
02007512STR
02007514LDR
02007516BL
0200751AADD
0200751CPOP

LoadFromNARC() subroutine at 0x02007508 (IPKS).

The only thing we would need, in order to allocate a file data in the RAM memory, is to call this subroutine early in the game. There is no need to call this function more times, as the memory region we will take over is never used by the game, and therefore our data will not be overwritten.

For early calling this subroutine and initialize its parameters, a branch instruction was placed in 0x02000C80 (Diamond and Pearl), 0x02000CB4 (Platinum) and 0x02000CD0 (HeartGold and SoulSilver) in each ARM9. All of these addresses are found in the Main() subroutine in each game, which is executed only once in the game, at the very start of its execution. These branches must direct the game to a custom subroutine where R0 will be initialized with 0x023C8000, and R1 and R2 will be initialized with the "synthetic overlay" file ID, before branching to the LoadFromNARC() subroutine.

Last, an unused file in the ROM filesystem of each game must be replaced by the file with the data that will be loaded in 0x023C8000. For Diamond, Pearl and Platinum, we can use data/weather_sys/weather_sys_9.rlcn (unused weather graphics). For HeartGold and SoulSilver, a/0/2/8/0.rlcn can be used (that NARC is a leftover from previous games).


Tutorial

The subroutine designed for initializing and executing the LoadFromNARC() function is the following one. However, depending on the game version, some bytes may change (specifically these involved in the loaded file ID or the LoadFromNARC() address). The different assembly codes for each game are specified in the table below, along with the offset they must be written in. Here is an example of how the subroutine is designed for IPKS:

00000000PUSH R2-R7, LR
00000002LDR R0, [PC, 0x14]
00000004NOP
00000006MOV R1, 0x41
00000008MOV R2, 0x9
0000000ALDR R5, [PC, 0x8]
0000000CBLX R5
0000000EMOV R0, 0x0
00000010MOV R1, 0x3
00000012POP R2-R7, PC
000000140x02007509
000000180x02FC8000

Initialization subroutine dissassembly for HeartGold and SoulSilver.
Game Code Initialization Subroutine Offset Initialization Subroutine Code Branch Offset Branch Code Loaded File
ADAE 0x021064EC FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD F1 64 00 02 00 80 3C 02 0x02000C80 05 F1 34 FC data/weather_sys/weather_sys_9.rlcn
APAE 0x021064EC FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD F1 64 00 02 00 80 3C 02 0x02000C80 05 F1 34 FC data/weather_sys/weather_sys_9.rlcn
CPUE 0x02100E20 FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD A5 6A 00 02 00 80 3C 02 0x02000CB4 00 F1 B4 F8 data/weather_sys/weather_sys_9.rlcn
IPKE 0x02110334 FC B5 05 48 C0 46 1C 21 00 22 02 4D A8 47 00 20 03 21 FC BD 09 75 00 02 00 80 3C 02 0x02000CD0 0F F1 30 FB a/0/2/8/0.rlcn
IPGE 0x02110334 FC B5 05 48 C0 46 1C 21 00 22 02 4D A8 47 00 20 03 21 FC BD 09 75 00 02 00 80 3C 02 0x02000CD0 0F F1 30 FB a/0/2/8/0.rlcn
ADAS 0x0210668C FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD F1 64 00 02 00 80 3C 02 0x02000C80 05 F1 04 FD data/weather_sys/weather_sys_9.rlcn
APAS 0x0210668C FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD F1 64 00 02 00 80 3C 02 0x02000C80 05 F1 04 FD data/weather_sys/weather_sys_9.rlcn
CPUS 0x0210101C FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD B9 6A 00 02 00 80 3C 02 0x02000CB4 00 F1 B2 F9 data/weather_sys/weather_sys_9.rlcn
IPKS 0x02110354 FC B5 05 48 C0 46 1C 21 00 22 02 4D A8 47 00 20 03 21 FC BD 09 75 00 02 00 80 3C 02 0x02000CD0 0F F1 40 FB a/0/2/8/0.rlcn
IPGS 0x02110354 FC B5 05 48 C0 46 1C 21 00 22 02 4D A8 47 00 20 03 21 FC BD 09 75 00 02 00 80 3C 02 0x02000CD0 0F F1 40 FB a/0/2/8/0.rlcn
CPUJ 0x02100214 FC B5 05 48 C0 46 41 21 09 22 02 4D A8 47 00 20 03 21 FC BD E5 69 00 02 00 80 3C 02 0x02000CB4 FF F0 AE FA data/weather_sys/weather_sys_9.rlcn
IPKJ 0x0210F868 FC B5 05 48 C0 46 1C 21 00 22 02 4D A8 47 00 20 03 21 FC BD 21 74 00 02 00 80 3C 02 0x02000CB4 0E F1 D8 FD a/0/2/8/0.rlcn
IPGJ 0x0210F868 FC B5 05 48 C0 46 1C 21 00 22 02 4D A8 47 00 20 03 21 FC BD 21 74 00 02 00 80 3C 02 0x02000CB4 0E F1 D8 FD a/0/2/8/0.rlcn

In order to add the initialization subroutine in the ARM9, the initialization subroutine code sequence must be pasted in the initialization subroutine offset, depending on the used ROM. Also, for linking that subroutine with the Main() function, the branch code sequence has to be pasted in the branch offset. All the offsets are considered as RAM memory addresses, so in order to find the ARM9 address where these changes must be done, the 0x02 prefix must be ignored. In any case, the loaded file will be placed in the 0x023C8000 RAM address few milliseconds after the game is run.

To verify that the "synthetic overlay" is being loaded properly, DeSmuME Memory Viewer was used to inspect RAM memory in 0x023C8000 and in 0x02FC8000, the last one being one of the memory mirrors. The file is correctly loaded in that addresses, so thence it can be used for adding new data that doesn't fit in ARM9 or in the overlay files.


DeSmuME Memory Viewer inspecting RAM address 0x02FC8000 in ADAS, few seconds after the game is executed, showing that data/weather_sys/weather_sys_9.rlcn is loaded.

That new memory region can be used for repointing original ARM9 data to this region -such like map headers or the type effectiveness table- and therefore expanding them. As already mentioned, the maxium memory size recommended for loading data is 96 KB, large enough for placing such repoints or new custom subroutines.


Demanded credits

If you are planning to use this research on your project, credits for Mikelan98 and Nomura are needed. You can cite this research by copying the following reference in the credits section of your project's thread or webpage:

Mikelan98, Nomura: ARM9 Expansion Subroutine (pokehacking.com/r/20041000)

Total or partial reproduction of this research in other site is forbidden without prior authorization of the authors.






© 2020 PokeHacking

Pokémon characters and images belong to The Pokémon Company International and Nintendo.
This website is in no way affiliated with or endorsed by Nintendo, Creatures, GAMEFREAK, The Pokémon Company or The Pokémon Company International.