Need CRC calculation help

duffstone

Member
Joined
Sep 13, 2013
Messages
188
I've looked over the source code in the world creature editor, and I think one other CRC tool, but I'm failing to understand how it's working.

could someone here give me a good layman's overview of how the CRC is calculated in the code? I'm very green when it comes to bitwise operations and hash codes and stuff like that.

I can do all the .net work to rebuild a new and improved CRC editor, but since I don't understand how the CRC is calculated it's a moot point right now.

-Duff
 

Uli

Moderator
Staff member
Moderator
Joined
Aug 30, 2010
Messages
208
Application example:
http://www.zorc.breitbandkatze.de/crctester.c
Some information on what/how it does:
http://archive.info-mac.org/per/csmp/csmp-digest-v3-028.txt
 

duffstone

Member
Joined
Sep 13, 2013
Messages
188
Posted for notes as I hammer this out. I'll edit / delete / write this up when I figure it all out.

this will generate an SWG style CRC using VB.NET 2010.

dim crctable = new uinteger() {big list of 256 comma seperated crc's goes here}
dim c as byte
dim i as integer
dim crc as uinteger = &HFFFFFFFFUI

cbtExecute_click event:
for i = 1 to len(txtinput.text)
c=asc(txtinput.text(i-1))
crc = crctable(( c ) xor (crc>>24)) xor (crc<<8)
next
txtoutput.text = hex(not crc) 'hex output
txtoutput2.text = not crc 'decimal output

-Duff
 

Attachments

duffstone

Member
Joined
Sep 13, 2013
Messages
188
never mind, I got it! I needed to "not" the crc result, I'm guessing that's an inverse function but *shrug* either way it's working. LOL...

Now I need timbab's help in disecting the crc iff files so I can reconstruct them, then add my various functions. what does the STRT node do?
 

duffstone

Member
Joined
Sep 13, 2013
Messages
188
file notes: I'll edit / move when I get it all hammered out.

FORM (4BYTES)
RLONG (4BYTES) LEN TO EOF
CSTBFORM (8BYTES)
RLONG (4BYTES) LEN TO EOF
0000DATA (8BYTES)
RLONG (4BYTES) LEN OF SECTION
RLONG (4BYTES) NUMBER OF DATA ELEMENTS
CRCT (4BYTES)
RLONG (4BYTES) LEN OF SECTION
"CRC DATA GOES HERE" each crc is 4bytes REVERSED with no delimiter.
STRT (4BYTES)
RLONG (4BYTES) LEN OF SECTION
"STRING START POSITION GOES HERE" each start position is a RLONG (4byte) starting from 0.
STNG (4BYTES)
RLONG (4BYTES) LEN OF SECTION
"STRING DATA GOES HERE" each string value is delimited with 00 at end of string.

I say Rlong for reverse long, but it might be a reverse integer *shrug* I'm not exactly sure what a reverse 4byte number is. Still learning.
 

Timbab

Administrator
Staff member
Administrator
Moderator
Joined
Oct 6, 2010
Messages
1,057
Location
Magna Germania
Why or how do you want to recreate the CRC tool that's out already?

Form and chunk names are guess work, most of the time, but I assume;

STNG = String

STRT = Start

I assume RLONG might be a reverse integer, but hard to tell. Give me the name of an example file, so I can check it out.
 

duffstone

Member
Joined
Sep 13, 2013
Messages
188
meh, RLONG is misleading as it's not in the file, it's what I'm using as a place holder for the 4 byte reverse long / integer that follows the key names. I mostly posted that for my own benefit over the next few days as I continue the project.

Basically the CRC tool(s) we currently have work fine, but they're not really that great in practice. One gives you a nice UI for adding and saving, but you can't delete, sort, find, or create a new file from scratch. The other program just has a combo box, is very confusing to use, but at least it will let you delete entries. The other is just a simple CRC generator that doesn't do anything other than that...

so if you added an entry but didn't put your I before your E and there's no C... and you didn't know you did that... How do you find your error without knowing (writing down) the crc you added before you added it... finding mistakes is a huge time waster for me right now, I'd like to fix that. :)

shouldn't be hard now that I think I have the Iff file format understood and I can generate CRC's now with ease. the rest is just Interface and i/o stuff, pretty easy just need time to finish.

-Duff
 

NelkQuyiter

New Member
Joined
Aug 6, 2013
Messages
15
Yeah you are right, iff format is

"FORM"
Length //Net-Order Int
{ //Node List
"XXXXXXXX" //len 8 node name Asci
Length // Net-Order Int
Data // char[], FORMs will have other subnodes in data typically
}

If you want any tips let me know. If you are using .net, I have some code that might be of use to you(super old though, it gets the job done). Last I remember, I had a pretty decent OO class heirarchy setup where all you need to do is inherit the IFFNode and implement reading and writing and the IFF will handle saving itself (used it for .msh viewer, generic .iff editor, .ws editor, .trn editor, and a few other server related projects)
Best of luck friend :) all about the journey
 

duffstone

Member
Joined
Sep 13, 2013
Messages
188
It might help to get some of that code to use as examples, but I'm slowly plugging away at the IO stuff now.

so far I was able to create a custom crc.iff file from scratch, so my export code is working fine and tested with the client / server. small scale mind you, but it worked nonetheless.

This afternoon I was able to write a simple importer to grab the strings, which works, but it's super slow using filestream. But I can grab the strings out of any of the crc type IFF's now and load into an array or list or whatever. I dont' think it's worth my time to parse the iff for the actual crc's, I can calc those from the strings I've already parsed, then get them into a UI element that works best (datatable, listbox, something), and I'll be close to recreating the existing crc tool.

Still going to take a few weeks though I bet to figure out how to do it, how to do it faster, then how to debug it. :)

- Duff
 

duffstone

Member
Joined
Sep 13, 2013
Messages
188
ok, how exactly do the existing programs load their data? right now I'm doing it byte by byte, but there's got to be a faster way. I sped it up a bit by loading the file into a byte() array. but I still have to read the bytes to find the 00 delimiters and such.

so ... anyway still working. :) I'm making great progress but it's just super slow at this point. 1.5min to load the object template crc file. way to long.

-Duff
 

duffstone

Member
Joined
Sep 13, 2013
Messages
188
Ok, FWIW I have a completed tool that works the same as the existing crcedit.exe tool, but adds the delete function. The only real problem: It's supper slow on my P4 2.4ghz laptop. It runs "ok" on my i7 3.4Ghz machine, but not everyone has that much power.

I've been researching trying to find faster ways to work with array's, lists, tables, and other ways of storing data, but so far the fastest I can load the huge end of game TOC version of the object_template_crc.iff file is 26 seconds on my p4 2.4ghz laptop, or 6 seconds on my i7 3.4Ghz... Which seems forever really, and the existing crcedit.exe tool only takes a second or two...

anyway, the frame work is there, now I can quickly start adding desired features like merging files and such. here's the point where I need suggestions from all you, what would you want a CRC tool to do function wise?

-Duff
 

Timbab

Administrator
Staff member
Administrator
Moderator
Joined
Oct 6, 2010
Messages
1,057
Location
Magna Germania
Object_template_crc_string_table.iff loads in 6.59 seconds, but I got a i7 920, too.

Seems to do what it needs to do I think, though. Don't forget to attach the datatable to the FORM, so it resizes when you stretch the window, btw, also, make the FORM windows appear on top, or in the center of the tool.

In the edit, the 'Done' button confused me at first though, change it to like, 'Close', since that's what it does, essentially. Same goes for the 'Add', though you probably only need an overwrite button that closes the form.
 
Top Bottom