About SIE

Sytner

Administrator
Staff member
Administrator
Moderator
Sep 18, 2010
385
25
48
#1
About

This is a tool I made a few years ago when I was unhappy with the other IFF editors on offer. I wasn't going to release it publicly, but due to some friends getting significant use out of it I've put some work into updating and polishing it to be more usable by the wider community. The "IFF editor" nomenclature is historic as the tool now does a lot more besides!

In short, this tool:

  • Lets you edit datatables, palettes and IFF files (this is the format used by the majority of files in SWG, so you can edit most of the client in a generic way)
  • Has a .tre repository explorer with file history and previews for models and textures
  • Has an API which will allow other developers to build specialised tools for more file types in the future

If the rest of this post sounds confusing/complex to you - don't worry - it is aimed at two audiences at once. Anyone that has used other tools like Tre Explorer to edit or view SWG files in the past can use this in much the same way.

Features

Tree explorer with file history and appearance preview

The tree explorer includes a renderering window from the previous version of my world editor that lets users preview:

  • Appearance templates (.apt)
  • Static meshes (.msh)
  • Skeletal mesh generators
  • Animations
  • Component meshes (.cmp)
  • Portal property pages (.pob)
  • LODs (.lod)
  • Skeletal LODs (.lmg)
  • Objects (.iff)
  • DDS textures
  • Shader templates (.sht, basic display only)



Unlike most tree explorers in the past, the one in SIE works from a repository of .tre/.toc files rather than just one. This is configured by default for the PreCU publish 14 client but can be changed in the .cfg to match your needs.

The SIE tree viewer can also show/extract/open any version of a file contained in the history composed by the input .tre files.




Iff editor and template system

The IFF editor is the oldest part of the tool. It started out in 2009 as a simple replacement for the chunk editor in Tre Explorer, which was frustating me due to showing files incorrectly and corrupting them on save. It remained mostly unchanged until late last year when I decided to turn it into something more useful. The hex-editing functionality (a la Tre Explorer) remains but the highlight feature of this tool now is the template system.



The template system provides a syntax with which users can define the binary structures of IFF chunks. These templates can then be applied to chunks in order to edit their values or used to create a new chunk of that type. If named appropriately, templates are applied automatically on chunk selection. As such, keeping to this system for your own templates is highly recommended (the name will be generated automatically for you as a suggestion)!

For example, below I have created a template for terrain shader families and used it to create a new chunk. It has been given the default values specified in the template while everything else is initialised to 0.



Here I use the value editor to change some of the defaults that were set, the hex source is updated automatically.



If the template system looks scary then fear not - the community has already made over two hundred of them for you to use!

Template syntax and semantics (only read if you're interested in writing your own templates!)

The template system supports the following primitive types:

  • string, byte, bool, bool2 (2-byte wide bool), bool4 (4-byte wide bool), ushort, short, uint, be_uint (big endian), int, float, double, tag (char[4]), rtag(char[4] reversed)
  • 8-bit per channel colours: rgb, bgr, rgba, argb
  • 32-bit per channel colours: rgbf, rgbaf, argbf
  • vec2, vec3, vec4, quat, mat3x3, mat4x3, mat4x4
The primitive types can be used to create structures and arrays.

  • Arrays are defined like this Type[Name, ArraySize], where ArraySize is either a constant or a previously read scalar variable.
  • Arrays can be scaled (for example if you need to read a number of floats but the chunk gives the size in bytes) by Type[Name, ArraySize, ScaleFactor]
  • Arrays can read until the end of the chunk by Type[Name, inf]or Type[Name, -1]
  • Structs are defined like this struct[StructName]{Variable1,Variabl1}
  • Structs can be nested arrays can not
As an example of structs, vector types could have been implemented like so:

Code:
struct[rgb]{byte[r],byte[g],byte]}
struct[vec3]{float[x],float[y],float[z]}
struct[quat]{float[w],float[x],float[y],float[z]}
struct[mat3x3]{float[m00],float[m10],float[m20],float[m01],float[m11],float[m21],float[m02],float[m12],float[m22]}

As of version 3.0, you can create new forms from a template using the "Add Chunk from Template" button. You can append default values to POD types with the following syntax: float[x](0.43), string[y]("hello_swg"). Default values for vector types are not supported (I can add them if there's any demand for it). You can set the chunk type for the form to be added by adding a meta comment:

Code:
// META(CHUNKID:DATA)

If the meta comment is not provided, the program will attempt to find the appropriate value in the template file name. If this fails it will be inserted as CHNK.

As of version 3.2, you can define and use enumerations:

  • enum[theAnimals]{CAT=1,DOG=2,FISH=4}
  • int[animal](default=1,enum=theAnimals)

Datatable editor

The datatable editor supports all valid datatables in the .tre files (there are a couple of invalid ones with duplicate columns and garbage data). Unlike previously released editors for it has proper support for all types such bit vectors and linked enumerations.




Palette editor

The palette editor allows the editing of the .pal files found in the client. These control colourisations of armour and so on. The editor has a number of features to make this straightforward such as being able to create gradients and edit multiple values at once.



Other editors

The program also includes tools for:

  • Editing STF files
  • Searching all STF files (all/en/ja)
  • Editing CRC tables
  • Editing CEF files
  • Editing SND files
  • Editing WS files

The future and plugin API

The program directly makes use of functions from my main world builder's library such as those used to extract and view tree files. This means certain parts of the tool will get free upgrades as I work on the world builder. For example I may add .tre writing support and move to the new renderer.

Aside from adding "free" features like this and bug fixes, I've likely spent as much time as I can on this tool without seriously impacting progress on the world builder. However, the tool features an API which lets other developers create plugins to support new file types! Roughly speaking, the API requires developers to implement two simple interfaces providing:

  • Support for opening from a byte array (for opening from the tree viewer)
  • Support for opening from a file (for opening with start-up arguments)
  • Information about the status bar items required
  • Information about what file types it supports opening
  • Notifies the main window on open file name changes
  • Notifies the main window when the 'has changes' value of the open file changes
  • Notifies the main window when a status bar item value has changed
In summary, this means other tool writers need not waste time writing extensive UI code, .tre readers and so on. Timbab and I have some preliminary plans to merge our work so he can use this SIE as the base of his new version of Jawa Toolbox. The screenshot below shows a the beginning of a snapshot editing made using the plugin API in about 10 minutes:



Additional Templates

A number of template packs have already been created listed below. If you make any templates, please post them in this thread and I will update the OP!

Templates by Timbab

Templates for the vast majority of files.

Templates by Waverunner

Skeletal Appearance Templates (*.sat)
Logical Animation Table (*.lat)
3 chunks relating to hardpoints and distances for Detail Appearance Templates (*.lod)
.iff transform mask files (only one chunk)
Skeletal Mesh Generator Templates (*.mgn)
Skeleton Templates (*.skt)
Structure Footprints (*.sfp)
Asset Customization Manager


Templates by Phoenix

Terrain

Download: SIE 3.7.0.95

Latest release date: 2017/05/31
 
Last edited by a moderator:

Phoenix

UI Guy
Staff member
Moderator
Sep 25, 2010
204
9
18
27
#5
RE: Sytner's Iff Editor

No. It's just another tool just like tre explorer but organizes forms/records easier to read. And it allows for easier editting.
 

Phoenix

UI Guy
Staff member
Moderator
Sep 25, 2010
204
9
18
27
#13
Templates for Sytners .iff editor

These are the templates I have done so far. This first release is going to be .trn templates to help people understand how the .trn files are setup.

To install these just unzip them and put the terrain folder in the "template" folder of the iff tool.

https://www.dropbox.com/s/u4bmts3xsb8r3ur/terrain.rar?dl=0

Any questions you can message me on #modthegalaxy in irc.

Also here is a quick reference guide so you don't have to look at the templates.

https://www.dropbox.com/s/uno0i3gd1nm7fmv/Star%20Wars%20Galaxies.docx?dl=0
 

Sytner

Administrator
Staff member
Administrator
Moderator
Sep 18, 2010
385
25
48
#14
RE: Sytner's Iff Editor

Minor new version uploaded with a couple of small features and templates for world snapshots. I've also adjusted the OP to have a section where I can link to the templates people post in this thread. If you post something I'll probably see it and just add it in but just drop me a PM if I miss it!
 

Phoenix

UI Guy
Staff member
Moderator
Sep 25, 2010
204
9
18
27
#15
RE: Sytner's Iff Editor

Yup. Snapshot nodes and OTNL are both covered in the stock templates. Everything you need to edit them now. Any questions please refer to us in #modTheGalaxy chat in irc. More templates to come later. Main templates we can not do: Object .iff (Not feasible for iff editor. too may variables). Also .trn templates translate over to the .lay files.
 

Timbab

Administrator
Staff member
Administrator
Moderator
Oct 6, 2010
1,044
38
48
28
Magna Germania
#17
RE: Sytner's Iff Editor

Feck it, might as well start adding my templates then, was originally gonna release them all with Jawa v2.0. It was mostly a joint effort by Syt and I last year, but are only being now converted into templates. They'll be optimized for 2.7.6+.


Current Templates (435 Chunks covered):
  • Template - abstract/ .IFF
  • Template - animation/ .IFF
  • Template - camera/ .IFF
  • Template - chat/ .IFF
  • Template - Client Data File - clientdata/ .CDF
  • Template - Client Effect - clienteffect/ .CEF
  • Template - cockpit/ .IFF
  • Template - combat/ .IFF/.MKR
  • Template - creation/ .IFF
  • Template - Customization ID Manager - customization/ .IFF
  • Template - Slot Rule Manager - customization/ .IFF
  • Template - Effect - effect/ .EFT
  • Template - Game Music Manager - scene/ .IFF
  • Template - interiorlayout/ - .ILF
  • Template - Lightsaber Blade - appearance/ .LSB
  • Template - cell_lights - misc/ .IFF
  • Template - Mesh String Pack - appearance/ .APT/.SAT/.LOD/.LMG -- Most chunks aren't touched, only the ones to modify the important strings.
  • Template - Planet - appearance/ .PLN
  • Template - playback/ .PST
  • Template - Portal Object - appearance/ .POB
  • Template - Shader - shader/ .SHT
  • Template - Sound - sound/ .SND
  • Template - Sprite - appearance/sprite/ .SPR
  • Template - Swoosh - appearance/ .SWH
  • Template - terrain/environment/ .IFF
  • Template - texturerenderer/ .TRT




Updates
[2015/04/23] Following Changes:
  • Combat Directory IFF MKR - Fixed a problem with the .MKR Enum
  • Effect EFT -- Fixed some Metadata
  • Shader SHT -- Added Shader v1 support, fixed TSNS
  • Terrain Environment IFF -- Fixed CELS, changed the byte in DIST and TLOK to a boolean


Old Updates​
[2015/03/09] Following Changes and Additions (88 new chunks covered):
  • Camera IFF -- Added default, updated & completed
  • Client Data File -- Added 1 missing chunk (VGEF), updated & completed
  • Client Effect CEF -- Added default, updated & completed
  • Combat Directory IFF MKR - Added Enum support, added default, added tag support, updated & completed
  • Effect EFT -- Added Enum support, added defaults, fixed unknowns, added tag support, updated & completed
  • Lightsaber Blade LSB -- Added added defaults, updated & completed
  • Planet PLN -- New & completed
  • Playback PST -- Added Enum support, added defaults, added tag support, added 1 missing chunk (FIPR), updated & completed? Might be missing more chunks
  • Portal Object POB - New & fairly complete
  • Shader SHT -- Added Enum support, added defaults, added tag support, updated & completed
  • Sound SND -- Added Enum support, added defaults, added a template so it shows up for 3D, updated & completed
  • Sprite SPR -- Updated & completed
  • Swoosh SWH -- Added Enum support, added defaults, updated & completed
  • Terrain Environment IFF -- Added 3 missing chunks
  • Texture Renderer TRT -- New, fairly complete, missing some unused chunks
[2015/02/06] - All templates have been partially updated for v3.1 (Default values are still missing)

[2015/03/02] Following Changes and Additions (69 new chunks covered):
  • abstract/ .IFF -- Updated & completed slot_definition.iff
  • animation/ .IFF -- Added templates for posture_map.iff
  • appearance/ Lightsaber Blade .LSB -- New & completed
  • camera/ .IFF -- Updated & completed
  • chat/ .IFF -- Updated & competed
  • clientdata/ .CDF -- Added templates for the SDOE chunk
  • cockpit/ .IFF -- Fixed a typo
  • combat/ .IFF -- Updated and competed
  • creation/ .IFF -- New and completed
  • misc/ cell_lights .IFF -- New and completed
  • playback/ .PST -- New & completed (Might be missing some chunks, will have to double check for any unused ones)
 

Sytner

Administrator
Staff member
Administrator
Moderator
Sep 18, 2010
385
25
48
#18
RE: Sytner's Iff Editor

Great work Timbab!

Added a new version with several fixes and basic syntax highlighting for templates.
 

Phoenix

UI Guy
Staff member
Moderator
Sep 25, 2010
204
9
18
27
#19
RE: Sytner's Iff Editor

Good job bro. Going to have to make a list of ones that are done/not done. :|
 

Uli

Moderator
Staff member
Moderator
Aug 30, 2010
208
1
18
#20
RE: Sytner's Iff Editor

Phoenix said:
Good job bro. Going to have to make a list of ones that are done/not done. :|
hes just finished up them all last I heard.