The start of the gold rush….

The tool that saved the  Arena Depixelization Project (ADP)

Last post, I covered the Arena Font Editor, 1 of 3 tools that I use to edit TES:Arena graphics (and fonts). The font editor is part of the Arena Modding Suite by Hallfiry. The other part of that suite is the 2nd (set) of the three tools I’m going to write about.

Prior to the Arena Modding Suite, I used the method detailed in a previous post that was laborious and unpractical. Fortunately, this thread popped up on the Bethesda forums. And instantly my little “experiment” became a project and grew in scope. Were it not for Hallfiry, I would have surely abandoned it ADP before it every took off.

What’s it do?

The main functions of the Arena Modding Suite come in the form of the ArenaPacker and ArenaUnpacker. Rather than being a program that you work in, they are utilities that enable you to easily manipulate the game assets directly in Windows. Both programs directly work with the GLOBAL.BSA. Appropriately enough, one unpacks the entire BSA into a set of folders and the other takes that set of folders and packs it right back up although that is a bit simplistic view of what they do.

This slideshow requires JavaScript.

 

In reality, the programs not only work with the files but they also convert the files to the appropriate format. For ArenaUnpacker, when it extracts the files, it converts the non-standard IMG and SET files into easily edited PNG images. Additionally, it converts the INF files (map asset listings) to a text editor friendly format and SND files to WAVs (although I don’t have any interest in that part). ArenaPacker reverses the process and creates a packed GLOBAL.BSA based on files in the unpacked directory but doesn’t alter those files that were already extracted. This means I can have a working directory of all the files and my changes then “pack” my work-in-progress easily at any time to test in game.

 

All work is done from Windows

All work is done from Windows

 

Some notes about the Arena Modding Suite:

1. Quite a few of the images are compressed in a bizarro compression routine used by Bethesda and this software doesn’t have the functionality to uncompress them. No one had cracked that compression in all the years since the game was released (that is until very recently but more on that next time).

2. ArenaPacker is designed to compensate for using colors outside those available in palette file by converting non-palette colors to the nearest equivalent color in the palette. While it’s a handy feature, the images should be checked in game to make sure the colors aren’t changed to something wonky (as happened before I learned to use the palette tool in GIMP. If you stick to the exact palette (either by using a palette file or be just using colors in the actual images being edited), this isn’t a problem.

3. ArenaPacker is a little sensitive to what files are being repacked. When files are extracted, ArenaUnpacker creates a file list of all the files in each directory. This file list is used for when the files are repacked by ArenaPacker. So, if a file is missing or added that isn’t on the file listing, it will crash the program. So if I plan on “trimming” out the IMG/SET files not actually used, I’ll have to edit the file listing. However, it is very easy to fix so this isn’t that big of a deal.

- Martin

Alphabet Soup in Tamriel

Letters from long ago

After the recent work on the user interface, I decided to take a hack at changing the fonts. Arena fonts are stored as DAT files (the file extension that a lot of the text tables use). There are 9 separate font files and the game using them each in the game in different places (I.E. the character stat numbers are different than the travel menu summary). However, some of the text in the game isn’t from a font at all but part of a texture or image already premade (e.g. in the spell book, only the spell specifics is actually a font and not part of the image).

Spellbook

Only the spell specifics are a font

Click the font away

Thanks to Hallfiry’s Arena Modding Suite, I had the tools necessary. Hallfiry’s suite includes a separate program for editing fonts, called the Arena Font Editor. While the program isn’t the most elegant design, it does allow for editing of fonts in a fairly simple manner. The font editor allows for simple pixel checking and unchecking. Blocks checked will show and blocks unchecked wont. The size of each font letter can be set separately and while that size can be changed with the slider in the upper right corner of the editor, it should be noted that the game itself may not look good if the font size is too big.

Hallfiry's Arena Modding Suite

Hallfiry’s Arena Modding Suite

Hacking away

At first I didn’t understand how to use the editor. It turns out that in order to edit a font, the DAT file needs to be dragged and dropped onto the Font Editor. Additionally, there was no clear explanation on what the slider did. I eventually learned that it allowed resizing of each individual character in the font file (e.g. changing it from 5×5 to 6×6). I had already completed half of the font files when I discovered it’s purpose. The slide proves handy so that you can control the spacing between letters. In other words, you can have it one extra space wide so that the letters don’t touch. I did notice that not all letters were properly aligned to the left side of the box.

Pop-up text

Pop-up text

Testbed

I have somehow broke one of the fonts (my guess is that it’s out of range of what the engine can handle or maybe it just got corrupted). I have been using the same copy of Arena for a testbed since I started this project back before there were any  tools or this website. There are errant files and folders all over the place in the Arena directory (too include early BSA upackers, WinArena, and other crud). With this latest erratic behavior, I have decided to spend the grueling 5 minutes to download and install a clean copy. This way my efforts will match the end-user’s experience more accurately. Then it’ll be time for the second run through on the fonts to tweak the letters (and fix the broken font file).

- Martin

This slideshow requires JavaScript.

 

 

Polka dot shirts with checkered pants

Interface

One item that bothered me was that the interface elements didn’t really match. It seemed as each screen had it’s own style, particularly when you compare the “esc” menu and the inventory. Once I knew I could edit the inventory backgrounds, I wanted to make it match the other screens. However, I discovered that they use different palettes and I couldn’t find the right color that was on both palettes.

Inventory screen (original)

Inventory screen (original)

ESC menu (original)

ESC menu (original)

Compromise

I ultimately had to settle on as close as a match as I could get. Additionally, I tried to port over a few of the stylistic elements of the ESC menu into the other GUI elements to tie them together better.

- Martin

Dark green is as good as it gets

Dark green is as good as it gets

I used an alternate (unused inventory graphic) and tweaked it to add a little more character

I used an alternate (unused inventory graphic) and tweaked it to add a little more character

ESC menu (redone)

ESC menu (redone)

 

 

Sometimes it’s easier to walk around then straight through

Compressed

Quite of few TES:Arena IMG files are compressed in a crazy-wack-funky format that only the executable can decipher. No one has successfully found a way to decompress them. I had, quite a long time back, experimented with trying to replace them with uncompressed images but didn’t succeed.

Character screen

EXAMPLE: The character screen is two parts: stats on the left and character image/provincial background on the right. Each section is it’s own IMG file that is unfortunately undeciphered as of yet.

The “method”

I picked the “QUOTE.IMG” texture to edit because it was quick to find in-game and had clearly defined dimensions (full screen at 320×200). I loaded Arena in DOSBOX and waited for the quote to load (it’s the second screen after loading). Then I took a screenshot and since DOSBOX already uses native resolution, I didn’t need to alter the image. However, when I replaced the “compressed” version with the new one, it didn’t show up in game (it stayed black until the next screen loaded).

The screenshot version of the Quote image

The screenshot version of the Quote image

Wait there’s more

However today, Hallfiry (the maker of the Arena mod suite I use) revealed to me that it was indeed possible to do what I had attempted. Reinvigorated, I retried my previous method and almost instantly found a flaw in my method involving his program. Basically, his program translates uncompressed image files from Bethesda’s non-standard image format to PNG files for easy editing. When it does the conversions it creates two additional files, .MET and .PAL, containing color and format information to be used when reconverting back to the original format (repacking the archive). I failed to accountfor his program needing these files to successfully convert back the image. So this time I copied a .MET and .PAL from a similar “uncompressed” fullscreen image and renamed them both to match the “QUOTE” image.

Victory

This time it worked although it looked normal because I hadn’t edited it’s appearance. What this means is that all full screen IMG files could easy be redone using the “improved” method above. However, I wondered about partial full screen images such as the character screen.  Using DOSBOX debug, I quickly found that the character screen images and made a screenshot. NOTE: Dos debug shows what images are loaded so I also knew which image to replace. The character screen is composed of two images. On the left is the blue-ish stat background and the right is the province background with the character image overlaid on it. I chose the stat background due to simplicity. Using the method above, I only had to add one step. Knowing that the CHARSTAT.IMG covered only a portion of the image, I had to crop the DOSBOX screenshot to just encompass that side. I did some messy edits to test a few things and the results are below…

Now I can replace fullscreen graphics

Now I can replace fullscreen graphics

c

NOTE: I confirmed my suspicions that not only were the DONE and NEXT PAGE buttons were just part of the background, but the stat names and other misc “yellow” text was too.

Basically, this means I’m one step closer to a complete retexture. Fullscreen and partial fullscreen graphics are now replaceable to include: the title screen, the quote, all the image scrolls (yes this includes the text and I CAN fix the Uriel Septim typo), character sheets and backgrounds and what ever else is full screen. The only one I’m not sure about is the MAP. I’ll save that experiment for another day.

- Martin

Picking back up steam

Silence

I haven’t had a lot of time this summer but I have made tweaks here and there. There are a few textures I’ve done on another computer that I have yet to transfer over to my master files. Things have settled a little and I should be able to refocus my efforts on finishing the SET files as soon as possible.

Reenergized

For me, one benefit of extended “down time” on a project is that when I finally am able to return, I have regained a lot of my motivation. Additionally, I come back with “fresh eyes” and see ways to do things differently or better. While it’s a bummer to have to delay working on a project, I know I will always return because I love doing it and to me it’s very relaxing (even when I’m replacing every @#$% pixel in 1994 game with only 256 colors, 64×64 image dimensions and cryptic/bizarre file formats)

Pictures

Here are some photos showing some changes and things I am working or have noted.

-Martin

 

Bring marshmellows and a stick…

For this texture set, I wanted to retain the fire/lava vein effect in the walls. Although I had mapped it out from my previous run through all the textures as covered in a previous post, I ended up making quite a few changes. Click the picture below for a closer look.

Here’s a breakdown of the sequence of events from start to finish.

1. Create an outline – To do this I selected a dark color and outlined all the rocks letting anything outside the outline be designated for the fire/lava. I followed the source fairly closely but did take some liberties to make some a little bigger.

2. Fill in the rocks – I initially selected a tan color for the rocks since the original was largely tan-ish. I colored in every rock, one by one. There are easier ways but I enjoy it so I don’t mind using the color every pixel method.

3. Make tiling template – Since any texture in the set could be next to the other, I had to make sure that they match up naturally and didn’t have any obvious seams. I copied the first 3 left-side columns of pixels in the top most texture and pasted them on each image all the way down. Next, I repeated the process but for the right side. This made all 4 textures (this SET is 4 textures in a column) have the exact same sides. Since the top one was seamless, they now all are seamless.

4. Make duplicate of image in new layer – The duplicate layer is what I used to create a uniform and consistent lava pattern. To duplicate a layer, right-click on the layer in the layer toolbox, then select duplicate layer. An exact copy of that layer will be placed right underneath the original.

5. Remove fire/lava from top layer – First, I turned off the bottom layer and make sure the top layer is selected. Then, using the eraser tool, I erased all the fire/lava and miscellaneous areas not already designated as rocks (and thus colored in). This made the lava area transparent but it is still preserved in the bottom layer which is “hidden” from view when turned off.

6. Create lava layer – First, I turned the bottom layer back on and the top layer off. Then I started with the top image and hand created a gradient covering the whole image starting with yellow at the bottom and working to dark purple(ish) on the top. Then I copied this completed lava gradient and pasted it over the 3 bottom images.

7. Merge the two images – I “turned on” the top layer (making both layers on), then right clicked on the top layer in the layer toolbox. From there I selected “merge down” so the top layer and bottom layer become a single image with both the new rocks and lava together.

8. Revise – At this point, I decided that the lighter color rocks didn’t contrast well enough or give the lava the pop I wanted. I loaded another copy of the original texture in GIMP, picked out a new brown but then decided on a slightly lighter color than the original rock outline. Using the bucket paint tool, I filled in all the rocks with the new color. In this process, I also caught a few miscolored pixels and fixed them.

9. Create depth – Next, I added a lighter faded version of the rock color to the left side of all of the rock outlines. This created a highlight and adds the impression of depth to the rocks.

10. Touchup – Lastly I offset the image 1/2 on the horizontal plane so I can see how the whole file tiles sideways. Since dungeons and interiors don’t exceed 1 tile in height, I didn’t have to worry about this texture set tiling vertically. To offset in GIMP, press Ctrl + Shift + O and in the “X” box typing 32 which is half the total width of the texture. After clicking the “offset” button, the whole image will shift 32 pixels to the right and placed the edges of the image in the middle. I scanned and fixed any mismatchs or slight errors then shifted the whole image 32 more pixels returning it to its original place.

Icy walls of bleh

A little before and after pictures. There are a total of 5 ICE dungeon texture SETS. As you can see in the below images, they are mostly just recolored variants with different symbols. SIDENOTE: I presume they are Daedric symbols commonly seen in later games.

Here are three of the originals….

It took a bit to decide how much different, I wanted each to be. Ultimately, I just decided on to make them vary enough so that they would have their own appeal but maintain the overall theme. I haven’t seen these dungeons in game yet (never played far enough I presume). Once I complete all the SET files, I will do a test run through the game to see if I can find them (and others).

And the same there afterward…