CCG V3-1 Conversion:Part 13 Tools Guide

From NWN1 Custom Content Guide

Contents 1 Tools Guide 1.1 Torlack and Roboius' NWN Explorer 1.2 Bioware's Hakpak Editor 1.3 Modeling Tools 1.4 Bioware’s DDS Compression Tool 1.5 DDSTools 1.6 Tlkedit 1.7 GMax MDL import/export script

Tools Guide

This section provides a more detailed look at the available tools.

Torlack and Roboius' NWN Explorer

This file explorer comes in two versions (hopefully they will be reconciled soon!) The latest is from Roboius and supports SoU (http://nwvault.ign.com/Files/other/data/1055545441750.shtml) and a few other features while the original is from Torlack (http://www.torlack.com/index.html?topics=nwnexplorer_readme).

NWN Explorer lets you browse the NWN resource files (all of the models, textures, pictures, text, etc. that makes up the content of the game) and to extract them so that you can modify the files or use them as a base for making your own custom content.

NWN Explorer does not come with an installer so unzip the file into a directory and drop a shortcut to it on your desktop or into your NWN menu.

NWN Explorer replaces the older NWN Viewer with a lot more features including the ability to view models and other files in many different ways. In previous versions of this manual I recommended Zoligato’s NWN Viewer. This viewer is still available (http://nwvault.ign.com/Files/other/data/1026158289673.shtml).

Not sure which tool to use? Experienced models keep both around. NWN Explorer has a lot more features and it recognizes both patches and SoU content natively (NWN Viewer does not) so it is most commonly used. NWN Viewer does have two features making it worth consideration: it allows for mass export of files (select a range of files and export them all) and it recognizes talk tables (dialog.tlk).

Bioware's Hakpak Editor

Warning Another way to override Bioware content is to drop a file into the /override folder of your game directory. However, you should not do this. Hakpaks are much cleaner – they can group many overridden files into one package and, more importantly, they are specific to a module. Putting a file into the /override directory will override it for all modules you play on that computer which may have unintended consequences on other people’s games. In general, don’t use the /override directory. The only valid use of the /override directory that I would recommend is for temporarily and quickly testing your content without having to go through the process of updating a hakpak, saving it, etc. Once you have finished testing, don’t forget to load the new content into a hakpak and take it out of the /override directory.

A hakpak (a file with the extension .hak) is a package where you add custom files to either supplement or override standard Bioware content. Anybody who plays your module will need a copy of your hakpak so be sure to distribute it along with your module itself. You can use a hakpak to make Neverwinter’s standard goblin a different color (overriding standard content) or to add a new monster such as an Owlbear (adding new content).

Let me try to outline a few of the controversies surrounding hakpaks and their use in customizing NWN content.

First, let me note that hakpaks are the official Bioware method for distributing custom content. They are not free from issues or special considerations but they are the best way of getting your content out to the gaming public. People have come up with workarounds and lots of contradictory and nonsensical advice on what to do and what not to do with hakpaks. Let’s talk about a few of the issues.

1. The /override directory.
   • First, you can also override Bioware content by dropping a file into the /override folder of your game directory. However, this approach is not recommended. Hakpaks are much cleaner – they can group many overridden files into one package and, more importantly, they are specific to a module. Putting a file into the /override directory will override it for all modules you play on that computer which may have unintended consequences on other people’s games. Don’t do it. If you put custom content into the /override directory of your computer and wonder why other people’s modules don’t work on your machine, you probably have created the problem yourself.
   • Also, putting things in the /override directory like new .2da files will break the ability of the automatic installer to update your game to the latest release. To fix this you will need to restore the old version to the /override directory. You can find the old versions of these files in the /source directory of your game.
   • One .2da file per module is all that you are allowed. This means if you want to combine resources from multiple hakpaks it becomes tricky when you put similar content together from multiple sources because some of the model id’s and 2da files may overlap.
   • For example, you will need to re-edit and combine some of the files manually (specifically the .2da file). Also, Because of a very strict naming scheme that you have to fulfill to have the game recognize items and display them properly, there will never be a 'clean' solution. For details on how to combine hakpaks, refer to this tutorial on combining hakpaks: http://nwvault.ign.com/modules/tutorials/CombiningHakPaks.shtml. It is no longer strictly necessary to combine the paks physically because NWN allows multiple haks. However, the section on combining 2da files is still valid and required.
   • To give you an example, assume I create a bastard sword. I create it to fill the number 8 position. Not only do I have to name each file according to a set naming scheme I also have to ensure that the internal references to that object match the name. It is not even as simple as renaming the files. Internally there are references to my number 8 sword that have to be changed in roughly 5 places in at least three files. Nine files if you've include all three parts for all three colors. So roughly 45 to 54 internal references.
   • So say Poly creates a bastard sword. He has no idea I've done so and he picks the next available number which is 8 and builds his as the 8th piece. To use his and mine in one hakpak, one of us has to go back and change those references.
2. Official expansions may overlap with custom content.
   • Similar to the issue above, if Bioware creates a patch that includes additional content, there is a risk that it will interfere with your content. This problem becomes even more insidious because your module may work fine under version 1.22 and be broken under version 1.25. At least when merging other people’s content into your hakpaks you can test for incompatibilities. When it happens as an official update, anybody who has an old version of your module on their machine will be at risk. This is why I recommend setting ID numbers in .2da files to be much higher than the official ID numbers.
   • Expect even more of these issues when Bioware chooses to release expansion packs. Again, the best way to avoid these issues is to stay well above the range of ID numbers that Bioware uses or use the User ranges they set aside.
3. Official patches will not be in custom content released before the patch.
   • To override some of the files, like the .2da files, you take a copy of the entire file and add your own modifications to it. There is a risk if Bioware issues an updated version of that file in a patch that you will not have those updates in your file. For example, I dutifully used NWN Explorer to export the baseitems.2da file from NWN Main Data, to add a new weapon. But my game installation has a more recent copy of baseitems.2da in the new NWN Override file already. I'm on v1.30 of the game.
   • If I run a diff between the new version and the original one, there were two small changes (to the Heavy Crossbow and the Trapkit). Not good. Not surprising, in retrospect, but not good anyway. Why does this happen?
   • Bioware originally used the /override directory for its official version updates. As of version 1.28 it moved most of that content to patch.bif. NWN Explorer gets its data from either the original game or the new patches, depending on which part of the directory tree you go down. When I distribute an older version of a .2da file in a hakpak, Bioware’s changes disappear because my hakpak version takes precedence.
   • Practically speaking what this means is that when you are going to modify a current NWN file (e.g., a .2da file), look in \source\2dasource.zip first, before you go and export a file you want from NWN Explorer. Only if Bioware hasn't updated that file recently would you use NWN Explorer to export a new version from the original game installation.
   • If you don't follow these simple rules, you risk undoing Bioware's bug fixes when you distribute custom content. This same issue exists if Bioware distributes an update after you distribute your content but we let's not exacerbate the problem.
4. The dialog.tlk file cannot be put in a hakpak.
   • See more in the section below on this file. Bioware has indicated that they will be making a local dialog.tlk file that is hak pak-aware. This is likely to be implemented in Hordes of the Underdark.
5. Renumbering .2da entries.
   • If you delete rows from a .2da file in a hakpak, or even if you re-number them, it will mess up an existing module that used the previous version of your hakpak. The toolset put the entries into numerical order for you, but if you have used the hak in that module before and delete an entry, everything after that entry will be one appearance off. This is why you will notice entries in the standard .2da files that are not actually deleted, they are just blanked out using ‘****’ and other meaningless values. Use the same method if you are modifying a hakpak in the middle of developing your module. This also occurs quite frequently when combining hakpaks. And it creates problems if you load a saved .erf file that referenced a previous version of your hakpak. The only real solution other than leaving in blank entries is to recreate your module from scratch or change all of your model entries by hand.

Having said all of that, hakpaks are still the best way to manage custom content. The only way to better manage some of these issues would be to create a central registry for ‘reserving’ Ids, item numbers, etc. to ensure that nobody ever created another item that overlapped with yours. Even then the central registry would only be as good as the people maintaining it and registering their changes with it.

So with that behind you, let me touch on the basics of how to install your content into a hakpak and load that into a module. To create or modify a hakpak, launch the supplied nwhak.exe program (it can be found in the /utils subdirectory where you installed NWN).

1. Select File | New to create a new hakpak. Enter the name and other information on the main window (or use File | Open and choose the hakpak you want to modify). Do not modify any standard Bioware HAK files.
2. To add files select Resource | Add or simply drag the file into the list below.
3. To remove a resource from the hakpak, right click on it and select remove (you can multi select to remove several at one time by holding down <shift> or <ctrl>).
4. Select File | Save to save your hakpak. Save it into the /hak subdirectory where you installed NWN.

To use your hakpak you must add it as a resource to your module:

1. Launch the Aurora Toolset.
2. Open your module or create a new module.
3. Select Edit | Module Properties to modify the basic characteristics of your module.
4. Select the tab labeled Custom Content.
5. Using the drop-down combo box select your hakpak (if it does not appear in the combo box you probably have not placed it in the /hak subdirectory where you installed NWN). You can add multiple hakpaks this way with the topmost haks having higher priority (i.e., duplicate files in the higher haks override those in lower haks).
   • **** INSERT Figure ? ****
   • Figure ? - Adding a hak pak
6. Save, close and re-open the module. There seems to be a bug (at least in Release 1.26 of NWN) where it does not always load a new hakpak properly while a module is open. The only workaround I have been able to find for this is to close the module and then re-open it. This properly loads the updated hakpak data for most content (but not all).

Modeling Tools

If you really want to do modeling, go and download yourself a copy of gmax (http://www.discreet.com/products/gmax/) from Discreet Software. It is a subset of 3ds max for non-commercial game designers and it is free. You also need to pick up the GMax MDL import/export (http://nwvault.ign.com/Files/other/data/1029721132840.shtml) script to use it for NWN content.

The installation and usage instructions are fairly simple:

1. Download and install gmax.
2. Download and unzip the import/export script.
3. Drop the nwn_mdl_v<version#>.ms script from the zip file into the /scripts/startup directory of your gmax installation (not of your NWN installation).
4. Export the model you want to edit using NWN Explorer. Select ASCII model to export the model.
5. Start gmax. The NWN MDL Tools pane should start automatically.
6. Enter a filename and select Import. This should load the model.
7. Edit it to your heart’s content.
8. Select the base object.
9. Select Export from the NWN MDL Tools pane.
10. Select MAXScripts | Listener to show the listener dialog.
11. Copy and paste the decompile ASCII model into a text file. Gmax will only copy 10KB at a time so you may need to copy and paste multiple times.
12. You’re done!

If you don’t use NWN Explorer and need to decompile your model file into ASCII directly, use Torlack’s NWNMdlComp (http://www.torlack.com/index.html?topics=nwnmdlcomp_readme) model compiler/decompiler. You don’t need to perform this step if you use NWN Explorer to decompile models directly.

1. Unzip the NWNMdlComp executable. Drop a copy in your C:\WINDOWS or C:\WINNT directory (or put it in a directory and set your computer’s pathname to point to it).
2. If you have an existing model file to work with, run NWNMdlComp –d <modelfile> to decompile the binary model file from NWN. Don’t use the .mdl suffix. The output file will be named .mdl.ascii.

Bioware’s DDS Compression Tool

NWN uses a scalable texture system to increase system performance, known as the DirectDraw Surface (DDS) format. DDS files contain multiple versions of the same texture at different resolutions, called mipmaps. Mipmapping is a technique that improves image quality and reduces texture memory bandwidth by providing pre-filtered versions of the texture image at multiple resolutions. DDS files are also compressed.

What does all of this gibberish mean? It means DDS files are smaller and render faster on high-end 3d video cards. While not absolutely necessary, DDS files will give your objects that extra level of professional shine. It allows your hardware to switch between very hi-res textures when viewing an object close up while using low-res textures for faster rendering of the same object viewed at a distance. Computers with a graphics card that cannot handle DDS files require a copy of the .tga file instead (usually of a lower resolution so that they do not slow performance significantly). So if you use DDS files, include one at a lower resolution like 64 x 64.

Bioware has a tool called ProcessTextures.exe (http://nwn.bioware.com/builders/texturetutorial.html). It is a free, unsupported Windows utility but it will not work on all computers. It is dependant on your video card and drivers supporting the required compression. Most DirectX 8 drivers should work – it has been tested it on a GeForce3 and Radeon 8500, which worked fine, however the Matrox G550 cards do not work.

Bioware’s DDS Compression Tool will not uncompress a DDS file into a bitmap. It is one way only (.tga to .dds). To uncompress a bitmap, use DDSTools (described in the next section) and then discard all but the highest resolution bitmap.

A few things to be aware of:

• Bioware uses a slightly modified version of the DDS standard so you cannot use standard tools like Nvidia’s Photoshop plug-in to create them. You must use the Bioware tool (or DDSTools although I don’t recommend that program).
• DDS is a lossy compression algorithm but does a good job in most cases and the file sizes are much smaller and they run faster so don’t let that stop you from running them.
• If you absolutely must have one texture of extremely high resolution, put just the .tga in your hakpak for that file. This is generally not recommended though because those large .tga files really slow down even high end machines.
• Try doubling the size of your .tga file before compressing it to avoid pixelization. You will convert every pixel to 2x2 pixels (smooth it if you like); THEN convert it to DDS - quality should be better (although file size of course will be bigger).
• DDS compression works on 4x4 blocks of pixels. If you have more than 2 distinct colors in the same 4x4 block, you will have some pixelization (every 4x4 block is defined by 2 colors plus transitions between them). For most textures, it is really acceptable - you can show smooth transitions (4 level of transition for 4 pixel space - ideal) and you can show hard edges.
• All textures should be a power of two: 2X2, 4X4, 8X8, 16X16, 32X32, 64X64, 128X128, 256X256, 512X512 (the practical maximum that makes a difference).
• For really large models you may want to include several .tga and DDS files as textures (e.g., the Bioware dragon models use 3).

To use this tool:

1. Download the DDS Compression Tool (.zip, 262 KB), save it to a location where you can find it, and unzip it (using a tool like WinZip).
2. After unzipping the tool, you will find 2 files in the 'dds' folder (processtextures.exe and runme.bat) along with 2 directories labelled 'in' and 'out'.
   • **** INSERT Figure ? ****
   • Figure XX - Description
3. Place the .tga you wish to compress into the 'in' folder.
4. Double-click the 'runme.bat' in the 'dds' folder. This will convert the texture from .tga format to the compressed .dds format that is needed for the game. You will find the converted texture in the 'out' folder. It will have the same filename as the original, but with the .dds file extension.

DDSTools

DDSTools (http://downloads.neverwinternights.de/tools/ddstools.zip) is a freeware utility that will also compress and uncompress a DDS format file into a series of .tga files.

Bioware’s DDS Compression Tool will not uncompress a DDS file into a bitmap. It is one way only (.tga to .dds). To uncompress a bitmap, use DDSTools and then discard all but the highest resolution bitmap.

DDSTools flips your image vertically when creating the DDS from the bitmap (which will usually mess up how it maps to your model although you may not notice it if it is a random pattern). Use Bioware’s DDS Compression Tool instead (which does not flip) or manually flip your bitmap first. You can use TGAFlipper (http://nwvault.ign.com/) to do this. Also, Desoto’s will not compress 32-bit .tga files (i.e., files with an alpha channel) although Bioware’s will.

What all of this means is you need both if you want to use Bioware’s textures as a starting point for any of your models.

Tlkedit

Warning Usually you can avoid editing the dialog.tlk file for your custom content. There are different techniques for doing this: For some .2da files you can put ‘****’ in the string reference column and it will use the adjacent text column instead (e.g., appearance.2da for creatures). For many objects, you can simply override the object name in the Aurora toolset. When you do this the person who uses your module will never notice (e.g., baseitems.2da). Right now the only two sections of this tutorial that require dialog.tlk editing are for custom spells (which can be avoided as long as they are NPC spells) and custom races (using CODI’s Character Creator). This warning bears repeating. Do not edit dialog.tlk if you can avoid it.

Neverwinter Nights includes a data file called dialog.tlk. This file contains a huge list of thousands of text resources: all of the names of all of the objects in the dialog from the game. This is a common technique used to make it easier to internationalize a game – with all of the text strings in one place, translation to multiple languages is much easier.

Each resource in dialog.tlk has a number assigned to it called the String Reference. Unfortunately, editing dialog.tlk is fraught with issues:

1. It is a huge file (7 MB+) which makes for a big download.
2. It is not something you can override through a hakpak. So for each module that you play with a new dialog.tlk file, you have to manually swap in the new dialog.tlk before loading the game.
3. If you update NWN through the automatic updater, it will not work if you have changed dialog.tlk. You will have to restore the old version first, then update the game. Of course, any changes that Bioware makes to dialog.tlk for an update will break your version.

These examples make changes to dialog.tlk very difficult. Most users do not like messing around with their game installation to run a module and will refuse to download modules with modified dialog.tlk files. Many users have only a basic understanding of computers and find issues like manually swapping dialog.tlk files back and forth to be challenging.

Bioware is aware of this issue. Patch v1.25 of the game removed the need to modify dialog.tlk for some objects. For example, in appearance.2da if you now leave the String Reference field blank the toolset will use your label as the title for the object. This is a huge improvement and we continue to lobby the Live Team to make more patches like this.

The summary message of this section is this: Modify dialog.tlk at your own risk. It will certainly turn many people off from using your custom content. It is only necessary in a few circumstances so ask yourself if it is worth it before producing custom content that requires these changes. Here is my opinion on the subject, by section:

1. Weapons - it is unnecessary to modify dialog.tlk. You can edit the item name and description and stats in the Aurora toolset and distribute custom .erf files.
2. Portraits – not required.
3. Voicesets – not required.
4. Items without Models - it is unnecessary to modify dialog.tlk. You can edit the item name and description and stats in the Aurora toolset and distribute custom .erf files.
5. Armor and Clothing – not required.
6. Spells – required. Unless you want to use them only for NPCs. Again, if you are a module builder, ask yourself whether those custom spells are really critical to your game.
7. Creatures – made unnecessary with patch v1.25 of the game.
8. Custom Races – required. It is currently unavoidable for custom races. My only caution here if you are a module builder is that I would evaluate whether or not a custom race is really critical to the module before doing it. For the City of Doors Initiative (a Planescape mod) it is. For many other modules it is probably not and it will limit your target audience.
9. Visual effects – not required.

GMax MDL import/export script

Wayland wrote the original scripts used to import ASCII model files into NWN and export completed models back into ASCII model format. This section used to include a series of patches to those scripts to fix some known bugs. Instead, Joco has updated Wayland’s scripts (http://nwvault.ign.com/Files/other/data/1062667036829.shtml) to include these bug fixes and a new capability of editing emitters in Gmax.

---

« The CEP | Tools Guide | 2da Reference »

Main Page | CCG TOC