3DMAX MSH Exporter

[rosuto]

SWG 3D MAX MSH Exporter​

History

I started this project to prepare myself for a 3d modeling class I am taking this fall. I knew nothing about 3d modeling or 3d max, but decided if I was going to start learning, might as well create models I might be able to enjoy. I settled on SWG as the models are quite old, easy to look at, and I might be able to improve on some of them. One big problem though, there wasn't really a way to get them into SWG. I tried the blender extractor, which was ok but it didnt have any texture support. So after a week of swaying back and forth between 3dmax and the blender msh extractor, I decided to stick with 3dmax. Thanks to Xunil's work, recreating the MSH file wasn't that bad, but then I started to include the SHT file into the extractor. Very little information is publicly known about it, and while I could base a guess on one sht file of how they should look, I decided to increase my odds. I wrote a file parser to export certain sections of code for over 6000 SHT files relating to MSH files. So atleast now I have an idea of what is there. I just need to figure out what the heck it all really means. And that is where you can help.

Download Alpha 1.42​

Included

• MSH File Outline
• Multi Object Support
• Multi Texture Support
• Advanced SHT Creation

Not Included

• Attachment Support
• 32 bit face Support
• SIDX
• TFNS

Current Issues

• Lots of testing needed
• Lots of unknowns
• 20 object limit for models. Will boost later on
• 65,536 texture vertex limit per object

1.42 Updates

• Create APT files
• Added main SHT file creator selector switch. Will check all on or off
• *Extended shader list did not make this version

3D Max Support

• 2013

If a version is not listed above, it means no one has confirmed if it works or does not.

Installation and Running
To get started, download the file and save it to your computer. Any where works. I would recommend the scripts folder though. For me thats --> C:\Program Files\Autodesk\3ds Max 2013\scripts

To run the program, make sure the model you want to export is opened and in mesh form. Then goto the menu bar and choose: Maxscript --> run script. Navigate to the SWG_MSHExporter and hit open.

The following should popup if everything is good.

• Export: Choose whether or not to export that object
• Create Shader File: Choose whether or not to create a SHT file for the object
• Shader Name: Name of the SHT file that will hold the texture information for this object (*.sht)
• FVF Code: Unknown at this time. Only 48, 52, 56, 68 bpv have selectable options

Change what is necessary for each object you wish to export. Once finished, hit the Export MSH button. Choose what to name the msh file and where to save it.

SHT File creator

If you selected to create a shader file then you will see the above.

• Options: All these are unkowns, but supposedly might change texture features like gloss
• Option 9: Probably do not need to touch as it relates to water
• Options 14-16: Usually the same value
• Type: Type of texture
• TCSS: Unknown, might have to do with texture layer. Diffuse is required, normally set at 0.
• AVRS: Unknown, might be similar to TCSS
• Name of Texture: Name of the texture file (*.dds)

To have multiple texture types that relate to one model, for example a diffuse and a normal map, each texture will need to have its own UV map. The UV maps must be setup the same but need to be on different channels. Would not recommend doing this right now unless you are into testing.

Once you are finished, hit the create SHT button. You will need to do one of these for each SHT file you wanted to create. SHT files will save in the same location as you saved the msh file. Only exception as of now is if you saved the msh file in your swgemu directory, then they will be saved in the shader folder.

Make sure your new files are setup properly. Assuming SWGEmu is setup as follows:

C:\SWGEmu\appearance\mesh <-- most msh models go in here
C:\SWGEmu\shader <-- sht files go here
C:\SWGEmu\texture <-- dds files go here

If you are using another emulator, then adjust as needed.

Once done, you are ready to boot up the game and see if your new model works.

 

[rosuto]

Bugs, glitches, and Crashes with 3d Max that relate to using the extractor

I you run into an error while using the extractor in 3D max, please post the following information

• Is it repeatable?
• What were you doing?
• What was the error message?
• If you have it set to debug, please also post that error message and stage 0 section.

I have 3D max xxxx, will you include support for it also?

I tried to keep the script as inclusive as possible, but there maybe minor sections of code that use newer standards that could easily be replaced with older sections. Let me know what the error was and I will look into it

My game crashes after loading/when I get close to my model

Remove the model and see if that fixes it. If yes, then something is probably wrong with the msh file. This is usually the results of a file pointer being off or something else not agreeing with the rest of the model. Send me the msh file if you can and I will look into it.

I put my model in game and cannot find it
Make sure the objects size and origin is similar to the object that you are replacing in game. Also make sure that an effect that actually exist is listed in your models SHT file.

My model is green
Your model does not have a texture linked to it. Make sure the model has a SHT file linked to it, that file exist and has a real texture linked to it, and everything is in the right folders.

Model Conventions

One thing to know is that you cannot add new models/objects to the game with a client side mod. The server tells you where object xx is, so you cannot place new things in game. What you can do is replace current models by naming your new model that same thing and placing it in the correct folder. You also cannot change collision information as this is held by the server. So while you can make a house into a giant room, you will still run into invisible walls.

Naming Conventions

Only msh files need to be the exact name of what they are replacing. SHT and texture files (*.dds) can be named anything you want. But keep in mind that if you want to release something, please try to keep your mods seperate from other people's mods to prevent over writing. For example, if you make a new chair, do not name the the SHT chair.sht and texture chair.dds. Better to do something like rosuto_tatt_chair_wood.sht

BPV and FVF
BPV - bytes per vertex. It tells how much data is store for each vertex. The known numbers are 32,36,40,44,48,52,56,60,64,68,72. Mostly they tell you how many textures there are and if the object contains a vertex color. An object cannot contain any more than 6 textures (72bpv) or 5 textures and a vertex color (68bpv). The normal standard is 32bpv or 1 texture and no vertex coloring. FVF codes are unknown, but definately deal with bpv.

Multi textures
SWG can support up to 6 textures of varying types for each object. Each texture will require its own UV map. To do this, create another UVW unwrap and change the map channel to something unique for that object. The one rule is that each UV map must contain the same number of verts. The placement of the verts does not matter though. The list of common textures are:

• Decal
• Detail (up to two)
• Dirt
• Emissive
• Environment
• Hue
• Look Up
• Diffuse
• Normal
• Screen
• Specular

You can only have one of each except for detail which allows up to two (A and B)

Lighting
SWG will automatically light a model the way it thinks the model should be lit depending on its shape. If you do not agree with the job it has done, you can over ride this by creating your own lighting map. This is done by adding a vertex paint modifier to your object. Lighting is done in grey scale. White is bright and black is dark. It is also possible to use colors, but this adds a tint to your object, and usually ends up a color inbetween the texture and the vertex paint. To see the results properly in game, make sure to choose a C coded effect.

Effects
The effects tell SWG how to properly display a model in the game. Choosing the wrong effect will cause the model to either not show up at all, or to have a different texture look than desired. The effects come with 3 base codes. A, C, and H. A is for models that do not use vertex painting. These models will be rendered with hard coded lighting effects, hence they usually do not reflect light in game. C effects are for models that use vertex painting. H effects are unknown, but appear to be similar to A. C models will also reflect light in game depending on their vertex paint color. For information more information on what effects go with what texture codes, click here

 

[Aigik]

I'm installing now, thanks so much for this.
[hr]
Question... how do you select a texture type in the shader creator? All the fields are greyed out, and I can't select the drop down box to create a diffuse map, for example.

 

[rosuto]

Whats your BPV setting on the model? (listed with the fvf code) Because if its 32, then diffuse should automatically show up because its the only thing listed.

 

[Aigik]

Whats your BPV setting on the model? (listed with the fvf code) Because if its 32, then diffuse should automatically show up because its the only thing listed.

Nevermind, I got it... for some reason I was having that problem in max 2012, but it works in max 2010 for me. Anyway I was able to get my model into the game, but the UVWs don't seem to be translating over properly. I'll keep working at it to try to figure this out.

 

[rosuto]

Ok, I will look into 2012 to see if there was anything different about it. I will also continue writing the error codes to help solve issues a bit faster in the future, or give a clue that something is wrong.

As for UV, what you see in 3d max is what you should see in game. Or thats how it is supposed to work. I've been able to use any type of unwrapping without issues (2013). If you have fat ducks MSH importer, you can use it to check and see how things are exporting. Just import your MSH file and unwrap the uvw.

 

[Aigik]

All right, I've been playing around with this thing and here are some findings of mine. Thanks again by the way for making this, you have no idea how useful it is.

Anyway, here's a problem I've been having in numerous versions of max. I've tried it in 2010, 2012, and 2013. For some reason, certain models will not work properly with the shader editor, while others have no problem at all. Here's a screenshot of one of the problem models:

All of the texture fields are greyed out, and I can't select a diffuse texture. Here's a screenshot of my UVWs to show that I do have a UV channel:

Don't know what could be causing that, not sure if I did something wrong. I didn't do anything differently from other models that compiled properly. Here's the second problem I'm having. For some reason, certain models will give this lighting error:

As you can see, the prop looks fine in max:

No idea what could be causing that.

 

[rosuto]

Two things I want to say first

1) Great models, I really enjoy looking at them and am jealous
2) Thank you for taking the time for testing and finding bugs.

I have found a bug that causes the type field in the shader creator to remain blank. I was getting decimals when I only expected integers. Im also going to throw in a work around should it happen again, but need a bit more time to finish coding it. Hopefully by tonight I should have a new version up.

As for lighting, I totally forgot about that and thank you for bringing it back to my attention. I will keep looking into it but it seems to be hidden in the msh file, either that or we are screwed :-/ One thing that I did notice is that lighting seems to be based on the model type and not the location. For example the bottom half of a cube will always be darker than the top half.

 

[rosuto]

Update on the lighting... using vertex color effects the lighting of the object. It uses grey scale, so white is bright, black is dark. If you do not use it, then the game will try to light your object how it sees fit.

Updated to 1.25 which fixes the bug with the sht file creator.

Also found a bug with box and sphere bounding. When trying to build rooms, the size is slightly off thus allowing you to see through the walls. Current work around is to use tre explorer and copy the previous models box and sphere coordinates until I can eliminate it.

 

[Aigik]

I can't seem to get the vertex color solution to work... How are you changing the vertex color? I tried the vertex paint modifier, and I tried the "Assign vertex color" option under Utilities, but the model still has the problem in-game. Is there something I have to do specifically to get the game to recognize that I used vertex colors?

Also, another thing... I don't know if it's a problem with the exporter, or something I'm doing.... but for a lot of objects (usually larger ones) the object will disappear if you look in certain directions/angles.

 

[rosuto]

To assign a vertex color, add vertexpaint to the mesh in the modifier list. From there just to do a quick test to see if it is working: select the entire object, change the color to white, and hit the paint all paint bucket. This will make it very visible. To go back and create your own lighting, you will need to apply the colors as you want.

I will go and look through the maxscript some time later to see if I can pull lighting information for each vertex. If I can, then I will remove the coloring option and let people choose max or game lighting when exporting. Probably most of the day will be spent figuring out the issue with the bounding boxes though, which is probably why the larger objects are disapperaing on you. About half the data is good, the other half is slightly off of what it should be. Very annoying as I have already spent 3hrs on it and thought it was working 100%.

 

[Timbab]

Thanks for releasing it rosuto, I'll play around with it when I get a chance, no more mirrored and stretched textures, when I'll retexture them, hopefully!

 

[Aigik]

To assign a vertex color, add vertexpaint to the mesh in the modifier list. From there just to do a quick test to see if it is working: select the entire object, change the color to white, and hit the paint all paint bucket. This will make it very visible. To go back and create your own lighting, you will need to apply the colors as you want.

I will go and look through the maxscript some time later to see if I can pull lighting information for each vertex. If I can, then I will remove the coloring option and let people choose max or game lighting when exporting. Probably most of the day will be spent figuring out the issue with the bounding boxes though, which is probably why the larger objects are disapperaing on you. About half the data is good, the other half is slightly off of what it should be. Very annoying as I have already spent 3hrs on it and thought it was working 100%.

Yeah, that's what I was doing but the game still doesn't seem to act like anything's different, it still "generates" the bad lighting.

Anyway, thanks for the update! I'll keep working on my stuff and I'll start showing in-game screenshots (probably in another thread) once I can resolve the lighting issue. Timbab, what part of the game are you working on, out of curiosity?

 

[Timbab]

Been working on armor/weapons on and off.

 

[rosuto]

Aigik, if you want to send me one of the models that is giving you issues with the lighting, I wouldn't mind taking a look at it for you. Part of me is thinking that you might have something about your models that I am missing. Best case would be the max and the msh file so I can check the coding.

Other than that, still working on the bounding boxe and mesh requirement issues.

 

[Aigik]

Here's a download link to one of the models. Thanks for the offer.

https://dl.dropbox.com/u/15390227/con_gen_inorganic_gas.zip

Can't figure this out for the life of me.

 

[rosuto]

Ok, so good news and bad news. Good news is that I'm pretty sure that your model is good. Bad news is that lighting is still very much a learning process right now. From what I can tell, resource crates ignore lighting. I walked over one with my flash light on, ground lights up, resource crate does not. If I tell your gas crate to be a table, then light effects it, but of course the painting still doesnt work. So right now Im kind of like mmm..... back to the drawing board. The painting seems to be a very crappy way to do lighting anyways, as it may or may not work, probably because I still don't full understand it. Table works, crate doesnt, what gives?. One thing I have noticed is that models not linked to SHT files still get lighting effects, which means its either in the msh file format, or its somewhere else. My hunch is some where else as my cube test came back with results that don't fit its standard. Its from back in the day when I was trying to figure out uv maps.

Notice how only the bottom half of the cube is shaded and thus cuts through each triangle mesh.

Some other good news though
1) I have included error checking for mesh only. Will now disable objects that are not mesh. If you would like to include that object in your model, convert it to mesh, uvw unwrap it, and place a texture on it. UVW unwrap and placing the texture is because 3d max might invert a number of normals and I have found this to be the easiest fix for right now. If someone knows of a better method, just let me know.

2) Box maps are pretty much fixed, so no more seeing through models.

1.3 will be up some time tomorrow

 

[Aigik]

Ok, so good news and bad news. Good news is that I'm pretty sure that your model is good. Bad news is that lighting is still very much a learning process right now. From what I can tell, resource crates ignore lighting. I walked over one with my flash light on, ground lights up, resource crate does not. If I tell your gas crate to be a table, then light effects it, but of course the painting still doesnt work. So right now Im kind of like mmm..... back to the drawing board. The painting seems to be a very crappy way to do lighting anyways, as it may or may not work, probably because I still don't full understand it. Table works, crate doesnt, what gives?. One thing I have noticed is that models not linked to SHT files still get lighting effects, which means its either in the msh file format, or its somewhere else. My hunch is some where else as my cube test came back with results that don't fit its standard. Its from back in the day when I was trying to figure out uv maps.

Notice how only the bottom half of the cube is shaded and thus cuts through each triangle mesh.

Some other good news though
1) I have included error checking for mesh only. Will now disable objects that are not mesh. If you would like to include that object in your model, convert it to mesh, uvw unwrap it, and place a texture on it. UVW unwrap and placing the texture is because 3d max might invert a number of normals and I have found this to be the easiest fix for right now. If someone knows of a better method, just let me know.

2) Box maps are pretty much fixed, so no more seeing through models.

1.3 will be up some time tomorrow

Good news about the glitch being fixed. Anyway, I'll just keeping on working on models until a fix is made. I'll also be playing with the different shaders to see what looks best.

 

[rosuto]

Just found a huge issue relating to 3d max modifiers and my base extraction code. Depending on which modifier you last selected in each object will determine what gets extracted. I have found one fix for it, but I don't like it as it collapses all the modifiers into the object. Current work around is to select the very top modifier in each object before extracting.

What this effects: Everything. Most notably the model will be reconfigured or uv maps will be missing.

I will see if I can find a better way or at the very least, warn people to save the model before it is collapsed.

I have also added a wider mesh extraction window in 1.3. That should prevent most of the longer file/object names from covering up other things. If anyone has any gui suggestions, please let me know.

 

[Aigik]

Thanks for the new version, the disappearing issue is completely gone. Anyway, I've been trying to figure out how scaling works for things like detail maps. Specifically, I'm playing with an object that uses the effect "a_detail_bump". I extracted one of the structures from Tatooine and made some changes, but when I try to export it back into the game, the detail texture and normal map are both scaled up (probably a 1-1 ratio stretching them across the entire building). Obviously, that defeats the purpose, and the scaling for each of those textures needs to be reduced quite a bit. I've tried playing with all the options but they don't seem to do anything that affects detail or normal map scaling. The material shader I use (whether I use the one generated from your script, or use the default one in the game files) does NOT seem to affect it, so I am suspecting that this scaling info may be in the model file.

Oh yeah, when I was playing with the options numbers, I noticed option 17 seemed to affect model lighting in some way. Setting it to 0 made the model completely flat in lighting, and setting it to the max value made this weird glitchy skinny shadows on it.