question

Chat About Anything You Like - READ RULES
Post Reply
Tpe451

what do you guys use to repack games
NotGarry
Posts: 54
Joined: 20 Mar 2022, 15:07

It's too complicated to explain in detail. Basically, they use a program called FreeArc combined with other (de-)compression tools by several coders (e.g. Razor12911). The used tools depend on the game. Most repackers do it like this nowadays.
User avatar
Masquerade
Posts: 1728
Joined: 15 Feb 2021, 23:15

FreeArc is an archiving utility just like WinRAR and 7Zip.

However, FreeArc has the advantage of allowing you to include external compression methods to what FreeArc provides as standard. This effectively resigns FreeArc to being a simple host process for multiple different compression tools that we chain together.

There are three main types of external tools - precompressors, compressors and decryption tools.

Precompressors are responsible for decompressing input data. Most games nowadays use some kind of compression algorithm, deflate/zlib is arguably most commonly used but LZ4 is becoming more popular especially in Unity games. Furthermore, the Oodle family by RAD Game Tools is quickly becoming more mainstream through Unreal Engine. Even Ubisoft changed from LZO to Oodle in their Anvil Next 2.0 game engine (think Assassins Creed, Immortals Fenyx Rising and Ghost Recon Breakpoint).

One of the basic principles of data compression is that once data is compressed, you aren't going to be able to compress it optimally unless you decompress the data first. This is how the precompressor does its job. By decompressing the data, we then can apply more effective compression algorithms which is how we get ultra-small repacks. Such an example of a precompressor is XTool by our good friend Razor12911. XTool supports many different algorithms like the aforementioned Oodle, Zlib, Deflate, LZ4 and more. This wide range of compatibility means that XTool is the most commonly used precompressor as well as it is quick and multithreaded.

Other precompressors exist too for more specific purposes. The coder ProFrager wrote UELR (Unreal Engine LZO Recompressor) for Unreal Engine 3 games. Edison007 wrote AFR (Anvil Forge Recompressor) for Ubisoft older Scimitar/Anvil games that used LZO, not Oodle.

Moving onto decryption tools, they are very important. Not all games are encrypted, but when they are it's a huge pain. Encrypted data cannot be compressed so it's important that we decrypt that data in order to make the repack. Sometimes, the decryption tool can fit into the FreeArc compression chain so the installed game files will be encrypted again perfectly. An example of this is Razor's Death Stranding tool. Other times, all we can do is decrypt the files in the case of Monster Hunter Stories 2. The game still ran fine without encryption which is what made the repack possible. Another example of encryoted games are TellTale games. All of their games are encrypted with different Blowfish keys. Shegorat wrote TTGD which is a combination of decryption and precompression tool for TellTale games.

Finally, main compressors are ofyen the longest process of the FreeArc compression chain. SREP+LOLZ is the configuration of choice if you want the smallest size. SREP was written by Bulat Ziganshin, the creator of FreeArc and is responsible for removing duplicate data. Why bother compressing dupe data if SREP can use a huge LZ77 dictionary to remove and store those dupes, drastically cutting the size of data sent to lolz. For example, two thirds of precompressed data from Just Cause 4 could be removed by SREP. Just Cause 4 used a combination of three Oodle algorithms - Kraken, Mermaid and Selkie. LOLZ is a program made by ProFrager and is widely regarded as the best compression tool for game data. It has built in raw and header based stream detection for optimal compression of input data as well as support for large dictionaries and multithreading. LOLZ does have a function called LDMF (Long Distance Match Finder) which does a similar job to SREP - however I rarely use this so do not wish to comment on it. LOLZ compression is SLOW. Personally, my LOLZ configuration runs at between 1300kb/s to 1500kb/s depending on input data. You can speed it up slightly by turning detections off, but at that point you might as well just use a different algo. One overlooked but huge advantage of LOLZ is its ultra fast decompression speed. This reduces your repack install times! Because LOLZ compressor is slow for compression, some choose to use other algorithms such as LZTURBO or LZMA(2) which are faster but do not compress as well.

So here I have explained a little bit about the basic FreeArc compression tools and highlighted some of the tools that all repackers use. However to make more advanced repacks like we do here at KaOs, there's also a great deal of work done with custom tools outside of the compression chain to crank that compressed size down and compress everything as best as we can. There's also tools for game audio and Bink videos that I haven't discussed here. I'm sure if data compression fascinates you like it does myself, you won't hesitate to dig deep and build your research yourself.
I have PMs disabled here. If you would like to contact me, please do so in my Discord server.

Image
Post Reply