My Generic PSF (Portable Sound Format: PSF1, GSF) Ripping Strategy

Here I introduce my PSF ripping method.

Introduction

I assume you already know about the following things:

• What is PSF? - A format for video game music rips. In most cases, a PSF is a compressed game ROM whose non-sound stuff is removed by ripper.
• "Streamed?" "Sequenced?" What's the difference? etc. - Read PSF Frequently Asked Questions - Neill Corlett's Home Page
• What is pointer)? - For this purpose, I recommend you to learn C.
• Do I need to learn assembly? - At least you need to know basics. (for example: "What is a register?" "What is a stack?") If you already know one of assemblies, I think you can solve your problems during ripping by pinpoint googling.

Prerequisites

• Emulator (Debugger) - such as no$psx or no$gba debugger
  • It must have step execution function. (Step Into, Step Over, Run To Return)
  • It definitely should have a function that can change instructions by using assembly.
  • It should change register value. (If not available, use an external tool such as MHS.)
• IDA Pro - it will help you in reading assembly a lot. See also: IDA Pro - How To Load Game ROM
• Memory Scanner - such as Cheat Engine or Memory Hacking Software (MHS).
  • This tool is optional. It is handy when the emulator does not have built-in RAM search function.
  • It can be used to set a memory breakpoint, when the emulator does not have a memory breakpoint feature. (thankfully, nocash debuggers has the feature)

Typical structure of unaltered game

I think a game has routines like the following typically:

register int a0, a1, a2, a3;
register int sp;

void start(void)
{
    // Minimal Initialization
    sp = 0xXXXXXXXX;
    memset(BSS_START, 0, BSS_SIZE);

    main();         // will never return
}

void main(void)
{
    // Initialization Stage
    InitIRQ();      // set interrupt callback(s), an operation like this should exist in upper part of startup code
    a0 = 0xXXXX;
    InitFoo(a0);
    a0 = 0xXXXX;
    a1 = 0xXXXX;
    InitBaa(a0, a1);
    a0 = 0xXXXX;
    InitSound(a0);  // init sound registers
    InitMoo();

    // Main Loop Stage
    main_loop:
        MainFunc1();
        a0 = 0xXXXX;
        a1 = 0xXXXX;
        MainFunc2(a0, a1);
        a0 = 0xXXXX;
        MainFunc3(a0);
        MainFunc4();
        WaitForVSync();
    goto main_loop;

    // never return
}

// Callback for interrupts are registered at initialization stage.
// User callbacks are usually called by BIOS or CPU.
// Those callbacks are used for frequent and synchronous operations,
// so that music playback routine must be called by such a callback, in most cases.
// Some drivers may call the routine from a timer callback, instead of a vsync callback.
void VSyncCallback()
{
    UpdateSound();
    UpdateVideo();
    UpdateBlah();
}

// Start playback. This function is usually called from subroutines in main loop.
// Sound system needs to be initialized beforehand.
// "Load" functions may need to be called beforehand.
void PlayNewSong(int songId, ...)
{
    // initialize score pointer of each tracks, etc.
    ...
}

// About Load Functions
// --------------------
// The music data may need to be loaded from ROM to RAM before the playback,
// if the sound unit cannot read the data directly from ROM for some reasons.
// For example, PlayStation games need to load the music data from CD.
// They also need to load waveforms (PSX ADPCM) to the sound buffer.
// Most GBA games probably do not need it, because the processor can read ROM.
// For such systems, you may also need to find such load functions.

Details depend on the console and game.

Note: the code above is a classical C-style program. If the game uses C++ in object-orient manner, there could be more structures, dynamic memory allocations, and virtual functions (function pointer in C), then you would need more skills.

Strategy summary

1. Analysis before ripping
   1. Search score pointers
   2. Search "play new song" function
   3. Analyze loading routines
2. Ripping
   1. Minimize main loop
   2. Insert driver code (call the song select function)
   3. Minimize initialization
   4. Minimize callback
   5. Code refactoring

Search score pointers

I explained how, in Example Of Sequenced VGM Analysis.

Search "play new song" function

// Start playback. This function is usually called from subroutines in main loop.
// Sound system needs to be initialized beforehand.
// "Load" functions may need to be called beforehand.
void PlayNewSong(int songId, ...)
{
    // initialize score pointer of each tracks, etc.
    ...
}

Score pointers must be initialized when a new song starts playback. So that I always set a write-breakpoint to one of those pointers. (this is done by using MHS, because nocash debuggers cannot do that. In nocash debugger, you can set memory breakpoint by "Define Break/Condition" menu. Syntax for a write-breakpoint is [address]!! Read the help for details.)

Let me show the case of PS1 "Hokuto no Ken" as an example.

Set write-breakpoint and find where the score pointer gets changed

1. Run no$psx
2. Play the game until just before a song starts playback, then pause the game
3. Make a savestate (for quick redo, and prevent to change the song data address)

Here I need to set a write-breakpoint to one of the pointers, 0x80079B60.

1. Open "Debug → Define Break/Condition" from the menu
2. Enter [80079B60]!! (without quotes) and press OK

Now a breakpoint is set. The emulator will hit to the breakpoint immediately, after you unpause the game. In my case, it has stopped at 0x8003C00C. The pointer value gets updated by the instruction here, so I assume we can find the song load function by tracing back.

Note: Do not forget to remove the memory breakpoint before you start tracing back!

Backtrace from the write instruction

Repeat the following steps for 3-5 times:

1. Select "Run → Run to Sub-return" from the menu (or press F8)
2. If possible, read the function start address of previously noted instruction, from the one before the current instruction. (If not possible, use IDA Pro instead.)
3. Write down the instruction address

After that, obtain the function start address of each instruction addresses, by reading the previous instruction or using IDA Pro. Your note will become something like the following.

Sub-return	Function	Note
0x8003C00C	0x8003BF0C	Writes to score pointer
0x8003C1FC	0x8003C060
0x8003C3BC	0x8003C374
0x8003B84C	0x8003B714
0x80011F58	n/a	I guess this is not the playback function because its address is far away from others

I think one of function addresses is the top of playback function. Load a savestate and set a breakpoint to the top-level function (0x8003B714), then resume the game.

The execution will be stopped when a song starts playing. Edit one of the arguments to the function, and unpause the emulation. *1 If the game plays a different song, the function is apparently the song select function. It is what exactly we are searching for.

In my case, the game has played a different song by manipulating r2 register.

In actual research, you may fail and need to try for other arguments and functions. Try patiently, probably you will find the right answer after all (hopefully!).

After find the playback function, try to understand what each arguments mean.

Analyze load function

Games may need to load a music data from somewhere.

• If it's a PSX game, the game needs to load music data from CD
• If a music data is compressed or archived, the game needs to unpack it to another RAM area
• Some games may not have a load function (for example: GBA games, which can access to every ROM addresses directly)

Here we just need to do the same things. Try to know the data location, and use memory breakpoints.

In PS1 Hokuto no Ken, I somehow learned two facts:

• Music archive seems to be always loaded to 0x800FE000 (I decided to import music archive there by PSFLib)
• Music archive is unpacked by function 0x80012808

Minimize main loop

Most of functions (or all functions) in the main loop is not necessary for sound playback, since the sound playback is usually done by a callback.

The below is an example of main loop.

    // Main Loop Stage
    main_loop:
        MainFunc1();
        a0 = 0xXXXX;
        a1 = 0xXXXX;
        MainFunc2(a0, a1);
        a0 = 0xXXXX;
        MainFunc3(a0);
        MainFunc4();
        WaitForVSync();
    goto main_loop;

Try removing unwanted codes. Follow the steps:

1. Play the game until a song starts playing, then pause it
2. Set a breakpoint at the line like "call MainFunc1"
3. Run the game and it will stop immediately
4. Unset the breakpoint and change the instruction to NOP
5. Unpause the game and see if the music still work (If it works, the function call probably can be removed. If it does not work, the function may be necessary.)
6. Repeat those steps for every function calls in main loop

In my case, it has become an empty loop:

    // Main Loop Stage
    main_loop:
        //MainFunc1();
        //a0 = 0xXXXX;
        //a1 = 0xXXXX;
        //MainFunc2(a0, a1);
        //a0 = 0xXXXX;
        //MainFunc3(a0);
        //MainFunc4();
        WaitForVSync();   // this call can also be removed
    goto main_loop;

Insert driver code (call the song select function)

You should have small free code block in main loop. Use the block to patch the game to play a song immediately.

    // Main Loop Stage
    //main_loop:
        //MainFunc1();
        //a0 = 0xXXXX;
        //a1 = 0xXXXX;
        //MainFunc2(a0, a1);
        //a0 = 0xXXXX;
        //MainFunc3(a0);
        //MainFunc4();

    // Song Starter Example
    UnpackMusicArchive(MUSIC_ARCHIVE_ADDRESS);  // call loading functions before the playback function, if available
    PlayNewSong(SONG_INDEX);

    main_loop_hacked:
        WaitForVSync();   // this call can also be removed
    goto main_loop_hacked;

Reset & Run the game after inserting the code. Does a song start playing? Can you change the song by editing the arguments? If so, you have almost done your work!

For the next step, you should make a copy of the ROM file, and apply the patch to it. Then, open it instead of unaltered ROM by the emulator.

Minimize initialization

    // Initialization Stage
    InitIRQ();      // set interrupt callback(s), an operation like this should exist in upper part of startup code
    a0 = 0xXXXX;
    InitFoo(a0);
    a0 = 0xXXXX;
    a1 = 0xXXXX;
    InitBaa(a0, a1);
    a0 = 0xXXXX;
    InitSound(a0);  // init sound registers
    InitMoo();

Remove unnecessary calls like we have done in main loop:

1. Change the instruction like "call InitFoo" to NOP
2. Reset & Run the game and see if the music still work (If it does not work, reload the ROM, and try removing unnecessary calls in the subroutine.)
3. Repeat those steps for every function calls

For example, the final result may become like the following:

    // Initialization Stage
    InitIRQ();      // set interrupt callback(s), an operation like this should exist in upper part of startup code
    a0 = 0xXXXX;
    InitFoo(a0);
    //a0 = 0xXXXX;
    //a1 = 0xXXXX;
    //InitBaa(a0, a1);
    a0 = 0xXXXX;
    InitSound(a0);  // init sound registers
    //InitMoo();
    ...

void InitFoo(int a0)
{
    InitSoundRegion();
    //InitJoypad();
}

Minimize callback

Do the same thing to callback function.

void VSyncCallback()
{
    UpdateSound();
    //UpdateVideo();
    //UpdateBlah();
}

Code refactoring

Probably, patched main function is filled by a lot of NOPs. If you think that is not beautiful, you may want to create a new main routine.

Note: For PlayStation games, you can write the new driver code in C, thanks of PSF-o-Cycle.

// commented-out instructions are filled by NOP

void start(void)
{
    // Minimal Initialization
    sp = 0xXXXXXXXX;
    memset(BSS_START, 0, BSS_SIZE);

    main_PSF();     // will never return
}

void main(void)     // no longer used
{
    ...
}

void main_PSF(void) // inserted to an unused code block
{
    // Initialization Stage
    InitIRQ();      // set interrupt callback(s), an operation like this should exist in upper part of startup code
    a0 = 0xXXXX;
    InitFoo(a0);
    a0 = 0xXXXX;
    InitSound(a0);  // init sound registers

    // Song Starter
    UnpackMusicArchive(MUSIC_ARCHIVE_ADDRESS);
    PlayNewSong(SONG_INDEX);

    // Main Loop Stage
    main_loop:
        WaitForVSync();
    goto main_loop;
}

void InitFoo(int a0)
{
    InitSoundRegion();
    //InitJoypad();
}

// Callback for interrupts are registered at initialization stage.
// User callbacks are usually called by BIOS or CPU.
// Those callbacks are used for frequent and synchronous operations,
// so that music playback routine must be called by such a callback, in most cases.
// Some drivers may call the routine from a timer callback, instead of a vsync callback.
void VSyncCallback()
{
    UpdateSound();
    //UpdateVideo();
    //UpdateBlah();
}

Finalize

You need:

• Optimizing
• Timing
• Tagging

Notes

Assembly

General

• Depending on architecture, there are some pseudo instructions (macro) like the following (MIPS). IDA Pro will recover those pseudo instructions, however, many other disassemblers do not. (PSFLab and nocash debugger do not, at least)

li $t0, 0x1234ABCD
  ↓
lui $t0, 0x1234
ori $t0, $t0, 0xABCD

MIPS

• The immediate value of addi/addiu instruction is always signed.

addiu $t1, $t0, 0xFFFF
  ↓ means
addiu $t1, $t0, -1

• Jump and branch instructions have a "delay slot". This means that the instruction after the jump or branch instruction is executed before the jump or branch is executed.

addiu $a0, $zero, 1
jal $8001A000  ; a0 = 2
addiu $a0, $zero, 2

How to know vsync callback address

• PS1: Search xrefs to VSyncCallback by IDA Pro (It must be identified by PsyQ sigunature).
• GBA: Interrupt handler function address must be written in 0x03007FFC. The handler function must read 0x04000202 and see what interrupt has raised. See GBATEK for details.
• General: Repeat "Run to Sub-return" from subfunction of vsync callback

How to wait for vsync

• PS1: VSync runtime function (It must be identified by PsyQ sigunature).
• GBA: SWI 05h (IDA Pro will display it as "SVC 5"), VBlankIntrWait BIOS function, however, this BIOS function will never return when the callback does not update 3007FF8 (see GBATEK for details). If the game does not use any other interrupts, you can also use SWI 02h (Halt), it does not require a flag update.

Graveyard

Old method. It is not necessary in most cases, but it might be required for some limited cases.

Removed Section: Set write-breakpoint by MHS and get program counter (PC) value

This section is removed since nocash debugger has a built-in memory breakpoint feature.

First of all, run emulator and MHS.

1. Run no$psx.
2. Run MHS and open the no$psx process.
3. Play the game until just before a song starts playback, then pause the game.
4. Make a savestate, as usual.

I need to set a write-breakpoint to one of those pointers, PSX-RAM:$79B60. I assume it is already in the main address list. (See Example Of Sequenced VGM Analysis!)

Right-click on the address and click "Find What Writes This Address".

It attaches debugger to the emulator and opens Disassembler and Helper windows. Then delete the added address from Helper window.

Activate Disassembler window, then right-click on a random instruction and click "Breakpoints → Add Software Write Breakpoint Here".

An address must be added to Breakpoints tab in Helper window. Double-click the item to edit it.

Edit the address to watch the PC address of score pointer, then click OK.

Finally a breakpoint is set! Back to the emulator and continue the game. Probably the emulator will run very slowly, but please wait until something will happen.

Hopefully, the emulator will stop running, and MHS's Disassembler will popup and highlight the current execution address.

Now I need to read the program counter (PC), however, the emulator is freezing because of the breakpoint. Therefore, I have to read it from MHS. How? The answer is written in Memory Addresses Of Emulators. Since NO$PSX always has the current execution address in ESI register, I need to view the value through Registers tab in Helper window.

Of course it's PC address, so convert it to game address.

Finally, I get the code address 0x3C92C (0x8003C92C). We need to resume the emulator before using the address though.

1. Remove your breakpoint from Breakpoints tab
2. Detach debugger by Disassembler window
3. Close Disassembler window

Correction: In the screenshot above, I tried to search an instruction that writes to track 1. However, I needed to search track 3 in fact, because track 1 is used by SFX. I got 0x3C010 (0x8003C010) after the correct process.

*1:no$psx can change the register value. Right-click the target register column and select "Change Value" menu item.