Denrei Maya

From Coda Wiki
Jump to: navigation, search

This stuff is all old and out of date. I am currently using what is more or less a complete rewrite for my own work. Contact me if you are interested. I'll post things up here once my current game project is complete.


These scripts are still very Beta. Please have patience and let me know when you encounter problems! Thanks!

These scripts were written by Denrei to assist him in the tons of modeling, animating and texture work he knew he would be doing for CodaMod. The idea is that you can keep working on your original low res and high res objects, and have the changes automatically migrated to the point of exporting into Unreal. You can rapidly make tweaks to the meshes at any stage of the development, and see the end results quickly, without vulnerability to human error. For example, if someone were to create a model, and get it into the engine only to notice some flaws which required them to go back and move some of the UVs on the low res mesh around - this would mean going through the process of rebaking maps, reweighting the mesh and tons of little tasks along the way... which are confusing and tiring to deal with. These scripts allow small and massive changes to be propagated with the click of a button - rather what can end up being hours redoing the same troublesome tedious work.


Before we can begin, you'll need...

Resources you will probably want to explore...

(I see no reason why the scripts wouldn't work on any version of maya with python support, but 2008 is the only version tested. Let me know if anyone with a different version tries it out)


  • Place the downloaded ActorXTool.mll file in your Maya installation folder (ex C:\Program Files\Autodesk\Maya2008\bin\plug-ins)
  • Place the downloaded and denreiMaya anywhere you want to keep them.
    • Add this location to your Maya.env file (ex C:\Documents and Settings\UserName\My Documents\maya\2008\Maya.env).
    • The added line might look something like this: PYTHONPATH = C:/UT3Project/PythonScriptsFolder;.

You can now start Maya.

  • Open the script editor (Window -> General Editors -> Script Editor).
  • Select the Python tab, instead of the MEL tab. Copy these lines to into the editor:
import maya.cmds as cmds
import maya.mel as mel
import denreiMaya
  • Highlight them, and hit ctrl-Enter. This should launch the Denrei Window. If it complains about not being able to find the module, check that your Maya.env PYTHONPATH is set properly.

Once this works properly, try setting up a hotkey to open the Denrei Window more quickly.

  • Open the hotkey editor (Window -> Settings/Preferences -> Hotkey Editor)
  • Under Categories, scroll down the the "User" category. Click the "New" button.
  • Give it a Name and Description which are useful to you. Then, copy and paste the following command into the Command field. Click the "Accept" button.
python("import maya.cmds as cmds;import denreiMaya;reload(denreiMaya);;");
  • Finally, assign it to whatever hotkey you like. Denrei uses Ctrl-`. To do this, just check the "Ctrl" box, put "`" in the Key field and click the "Assign" button. Then "Save" on the lower left of the window and you should be set. This key combination will now reload or create a new Denrei Window every time.

Things to know when using Denrei Window

All of the information in the window is actually stored on extra attributes which are added to the persp camera's transform. Because of this, all attributes are saved within the scene. Every time the Denrei window is created, it reads from these attributes to build itself. If you are working on one scene, then open another leaving the Denrei Window open, make sure you quickly recreate the window so that it has the proper information.

Make sure you build your character with him/her facing down the Z axis, and the X axis to his/her left. Keep the meshes placed relative to the proper locators.

Denrei Window can often fill in values with defaults. If you aren't sure what to put in a text field, try clearing it and seeing what comes up.


This assumes you understand the Unreal PSA and PSK file formats. If not, check out the resources at the top of this tutorial if you get confused

  • PSA file name is the name of the file which will be created when the Run ActorX button is clicked with Export Animation turned on. You can change this name, but keep the ".psa" extension proper. If you want to change the folder it gets exported to, see the "ActorXExport" section.
  • For each sequence in this scene, click the "Add Sequence" button. Fill in the name, start time and end time separated by spaces, then click OK. This will add the sequence to the list of animations to be exported once the Run ActorX button is clicked with "Export Animation" turned on.
  • MirrorKeyFrames button will attempt to create a mirrored pose at the current time using all selected objects. If no objects are selected, the entire currently selected character set is used. The script assumes that objects are named with 'R_' and 'L_', and correlates the objects based on this. If things are ending up wonky, try setting a key frame on the entire character at the time you are attempting to mirror. Not that this operation only sets new positions and doesn't explicitly set any keyframes, unless you have Auto Keyframe set to "On".
  • CycleAnimation uses MirrorKeyFrames operation to create a mirrored, cycled animation (such as a walk or run). Set the start and end times of the source animation to be cycled. Note that these times indicate the source animation which will be duplicated and mirrored. So if the entire walk cycle should end up being 40 frames in length, and start at frame 0, your values will be 0 and 20. Do all of your animating in the first 19 frames, then the animation from the rig will be duplicated and mirrored onto frames 20 through 40, creating a clean cycling animation. Note that any animation after the cycled animation gets completely removed, so use with caution.


The first drop down set of fields labeled Description in this section are for the automatic creation of the UTCustomChar.ini file which will be needed to load any new character into Unreal Tournament 3. This process should be entirely automated unless your scene has more than one object of a particular PART_ID. More information can be found in the forums at Epic Games.

Denrei Window can help with the organization of a scene set up to create normal maps between high and low resolution character meshes. The set up relies on display layers to dictate which meshes are high and which are low resolution objects, and can then help you quickly build exported geometry in a way Unreal likes. At the moment, only ActorX is supported as an exporter, but Denrei took great pains to make sure that other exporters for other games could be added easily to Denrei Window in the future. Please let him know if you have any specific requests.

The first thing you'll need to do to use the modeling tools is allow Denrei Window to initialize the scene. This involves creating the display layers it needs as well as some helper locators which dictate the position of the various meshes in the scene. Go ahead and click the initialize button. Also check out the starter scenes available in the downloads section. These scenes come with prepared skeletons compatible with Unreal Tournament 3, and bound meshes which can be used as source for your own vertex weighting. Very helpful.

Any layer with a "_" preceding it's name marks that the layer contents are generated automatically and are not to be hand edited. Things in these layers get deleted and recreated frequently, so be warned. The created layers are as follows:

  • scrapLayer - Keep whatever you like in here. Denrei keeps this layer hidden during normal work. This is where the locator objects are placed by default.
  • lowResWorkLayer - This is the geometry which will end up getting exported. Keep objects here with names such as "Arms_01", "Head_02" and so forth. These names will be used during export to name the UPK files. Try to keep the history on this geometry to a minimum, delete frequently unless you really need it. Having weird history on these meshes can cause things to get confused. Also, its healthy to empty and re-add the objects to this layer every so often to keep it clean.
  • highResWorkLayer - Geometry which will end up being smoothed and used as the source in the baking of normal maps. Like the lowResWorkLayer, keep this layer clean and history free, if possible.
  • jointLayer - Keep your joints in here. Also keep in mind that actorX is VERY picky about the bones within a scene it is exporting. No duplicate bone names (even if its under a different hierarchy) and only 1 root joint is allowed.
  • weightSourceLayer - Bound geometry in this layer will have it's weights copied to the meshes which end up in the _exportLayer. Geometry here will have it's skinClusters merged if more than 1 is detected, but this is slow so keep around the ones which have already been merged. The weighted meshes do not have to have the same topology as the meshes being exported.
  • _highResProxyLayer - These meshes are created using the HighResMeshes button in Denrei Window. You may select individual meshes from the highResWorkLayer, or you may use the entire contents of highResWorkLayer by having nothing selected.
  • _bakeLayer - Automatically generated meshes which are triangulated and used as target meshes for the normal map baking operations. The contents of _highResProxyLayer are used as source objects.
  • _exportLayer - Automatically generated meshes which are properly triangulated, repositioned, bound to the skeleton using the meshes in weightSourceLayer as source. These meshes are the ones which will actually be exported by the Run ActorX button.

Keep in mind, a powerful feature of working this way is that you can freeze transformations of the meshes in their proper locations. Then move them a bit to be offset from the other meshes so they do not interfere with one another during the bake. The scripts will move all meshes to 0,0,0 when it creates the meshes to be bound and exported.

Also, every mesh in the lowResWorkLayer will try to mirror itself across the X axis so long as all verts in the mesh have a location of X >= 0.

Check out the SampleDenreiWindowScene and mess around with it to familiarize yourself.


This system allows the user to have complex Maya networks of PSD nodes - which get baked and exported to TGAs readable by Unreal. It handles things like flipping the Green channel of normal maps, and giving the textures proper UT3 names. This is not quite done yet, but it might work for you if you mess with it... :-P


These are the configurable options for exporting the contents of your scene all at once. The options are mostly self explanatory. Make sure that if you are exporting to Unreal Tournament 3, you have Reposition for UT3 turned on. If your scene has more than 1 root joint, make sure you either remove the unneeded ones or select Bake Animation. This will bake the animation on the rootJoint hierarchy, and remove all joint hierarchies other than the one selected for rootJoint. This is extremely useful if you are using a rigg where your bones to be exported are driven by non-bound bones (such as with Denny Lindberg's excellent Community Rigg).