How to expand the usable ARM9 memory data in IV Generation games: a synthetic overlay approach

Mikelan98, Nomura

Target games:

---

Introduction

Unlike every other stuff in the ROM images -like graphical, binary or sound resources found in the filesystem- the code files cannot be directly expanded, for they have an absolute location and size in the RAM memory. These files include arm9.bin, arm7.bin and the overlay files. Code files contain all the assembly code the game must execute for working properly, along with important binary data (map headers, starter Pokémon or the type effectiveness table, for instance). This means that, if we want to add new routines or expand the binary data, we must find unused memory regions in the code files, which are very infrequent and very short.

That's why finding a way to load code data in the RAM memory (and keep it loaded while the game is running) would solve some obstacles related with the arm9.bin/overlay files expansion issue. This strategy resembles the performance of the overlay files in the RAM, which are selectively loaded when needed, but in our case we would need a "expansion" file always loaded. For that purpose, we will have to find RAM memory regions that are never used by the game, so we can place new data there without the game overwritting it.

---

Research

Firstly, with the purpose of finding a region of free RAM memory where allocate our data, we used an unofficial DeSmuME release created for tracking unused memory addresses with ADAS (Pokémon Diamond, Spanish ROM), CPUE (Pokémon Platinum, English ROM) and IPKE (Pokémon HeartGold, English ROM). We set 0x10000 (65536) bytes as minium memory region size, and 0x02000000 and 0x02FFFFFF as search range (they are the main memory limits usable by the RAM, as GBATEK states).

Unused memory report for IPKE (Pokéon HeartGold).

Similar reports were obtained in the three games, so we took the four bigger memory regions and rounded them: 0x023C8000, 0x027C8000, 0x02BC8000 and 0x02FC8000, ensuring at least 96 KB of free RAM memory in each one for all the IV Generation games. However, we found out that the data written in these addresses are cloned between them, so everything written in one address will be quadruplicated in the four addresses. That means that we cannot use these four memory regions, but only one (as the others are simply "mirrors"). We chose 0x023C8000 for our purpose, because the other memory regions may be undefined in other emulators or devices.

The next step is to find a way for loading data in the RAM memory permanently. Instead of trying to create a new overlay file in the ROM filesystem (complicated and possibly buggy), we used a subroutine at 0x020064F0 (ADAE, APAE), 0x02006AA4 (CPUE) or 0x02007508 (IPKE, IPGE). This subroutine is used by the game for loading a specific file from a NARC to a specific address in the RAM memory, allowing us to load stable code and binary data in 0x023C8000. This file would be working as a "synthetic overlay", for it is a file present in the ROM filesystem that can contain both routines and binary data and write it to the main memory.

This subroutine (hereinafter named as LoadFromNARC() subroutine) takes three arguments: the first one for the RAM address where data will be loaded, the second for the NARC ID where the file that will be loaded is, and the last one for the file ID within the NARC. These three arguments are taken from R0, R1 and R2 respectively at the time the subroutine is executed.

The only thing we would need, in order to allocate a file data in the RAM memory, is to call this subroutine early in the game. There is no need to call this function more times, as the memory region we will take over is never used by the game, and therefore our data will not be overwritten.

For early calling this subroutine and initialize its parameters, a branch instruction was placed in 0x02000C80 (Diamond and Pearl), 0x02000CB4 (Platinum) and 0x02000CD0 (HeartGold and SoulSilver) in each ARM9. All of these addresses are found in the Main() subroutine in each game, which is executed only once in the game, at the very start of its execution. These branches must direct the game to a custom subroutine where R0 will be initialized with 0x023C8000, and R1 and R2 will be initialized with the "synthetic overlay" file ID, before branching to the LoadFromNARC() subroutine.

Last, an unused file in the ROM filesystem of each game must be replaced by the file with the data that will be loaded in 0x023C8000. For Diamond, Pearl and Platinum, we can use data/weather_sys/weather_sys_9.rlcn (unused weather graphics). For HeartGold and SoulSilver, a/0/2/8/0.rlcn can be used (that NARC is a leftover from previous games).

---

Tutorial

The subroutine designed for initializing and executing the LoadFromNARC() function is the following one. However, depending on the game version, some bytes may change (specifically these involved in the loaded file ID or the LoadFromNARC() address). The different assembly codes for each game are specified in the table below, along with the offset they must be written in. Here is an example of how the subroutine is designed for IPKS:

In order to add the initialization subroutine in the ARM9, the initialization subroutine code sequence must be pasted in the initialization subroutine offset, depending on the used ROM. Also, for linking that subroutine with the Main() function, the branch code sequence has to be pasted in the branch offset. All the offsets are considered as RAM memory addresses, so in order to find the ARM9 address where these changes must be done, the 0x02 prefix must be ignored. In any case, the loaded file will be placed in the 0x023C8000 RAM address few milliseconds after the game is run.

To verify that the "synthetic overlay" is being loaded properly, DeSmuME Memory Viewer was used to inspect RAM memory in 0x023C8000 and in 0x02FC8000, the last one being one of the memory mirrors. The file is correctly loaded in that addresses, so thence it can be used for adding new data that doesn't fit in ARM9 or in the overlay files.

DeSmuME Memory Viewer inspecting RAM address 0x02FC8000 in ADAS, few seconds after the game is executed, showing that data/weather_sys/weather_sys_9.rlcn is loaded.

That new memory region can be used for repointing original ARM9 data to this region -such like map headers or the type effectiveness table- and therefore expanding them. As already mentioned, the maxium memory size recommended for loading data is 96 KB, large enough for placing such repoints or new custom subroutines.

---

Demanded credits

If you are planning to use this research on your project, credits for Mikelan98 and Nomura are needed. You can cite this research by copying the following reference in the credits section of your project's thread or webpage:

Total or partial reproduction of this research in other site is forbidden without prior authorization of the authors.