Housing Progress:

Sep 13, 2013
188
2
18
#1
Ok, I managed to get the house spawned into the world. I've not yet solved the POB problem, but everything else is... well... close to ok anyway :)

So as you can see from the screenie, I'm seeing all the way through the house, no interrior is seen. but you can see through them. :) it's a start. This is using the non-windowed POB file for the same small house so that's how I got it zoned in.

I'm missing some textures I think, or something. I thought I got them all but it might be part of using the wrong POB, not sure. either way back at it.

but progress!!!

-Duff

*update*
it was a pixel shader causing the problem, so no more outside texture oddities. I still don't have the correct interrior yet but I'll work on that this weekend if time permits. gotta go to bed and take more cold medicine for the time being. - Duff

*update2*
Well I've reversed all the verts, I think I did it correctly but who knows. no way to tell really. I took the whole vert byte chunk (48 bytes total for each IDTLFORM), broke it out into 12 bytes (4 bytes per each x, y, & z, long or float shouldn't matter), took the 4 rows of 12 bytes and reversed the order, then pluged it back into the hex code. Including the PRTSFORM, there are 11 0000VERT chunks that I reversed. I changed the nodes to match the old style numbering, and no dice.

I also found some verts in the LOD file, so I reversed them as well and no dice. Client won't zone in. Guess I should give up and try something different tomorrow. Does anyone know if you can load the POB file with the OLD style LOD file? Or vice versa? sure would be nice if I could narrow down what file I'm having issues with. For all I know there's another file down the chain that's borked... Fun fun fun! - Duff

*update3*
Ok, found a much easier way to make this work. I'll delete all this junk and add a new guide when I have time to write it but for now it's like this:
1) you need to edit the LOD files. some buildings have more than one.
2) That's pretty much it. the rest is just making sure that all the necessary files are included for the client to use. Most are intuitive, some are not. I'll list them in my write up.

-Duff
 

Attachments

Sep 13, 2013
188
2
18
#2
ok, one thing I didn't do was to reverse the verts in the CMSHFORM in the cells area. But that brings up another question, what does the INDX mean? I've tried to logically think it through but for the PRTS's they don't seem to increment and they're identical on 9 of the 11 present.

I ask because there's a huge index that goes with the 0000VERT on the CMSHFORM and does that need to be reversed as well?
 

Uli

Moderator
Staff member
Moderator
Aug 30, 2010
208
1
18
#3
http://i64.photobucket.com/albums/h193/SWG-Uli/f98fe0dd.jpg

http://i64.photobucket.com/albums/h193/SWG-Uli/feba39f0.jpg

end results.
 
Sep 13, 2013
188
2
18
#4
Yeah, at this point it's just a matter of inexperience. I've isolated the issue solely to the POB file as near as I can tell. I can get the structure working fine with the OLD 14.1 POB file modified to use all the newer LOD's, Msh's, apt's, etc... so I know those files are working ok, and the inside LOOKS right, but the portals are not there so you don't get the desired effect.

Which tells me it's something with the portals since the cells and CMSHFORMS appear to be very close if not identical. The only real difference I'm seeing is an extra leading byte in each of the PRTL0004 entries, and that the 2d portals are being loaded as VERT & INDX instead just a flat PRTL data node.

Still working the problem, and may be focusing on exactly the wrong thing, but that's how you learn. Next step is to take the old POB, and add new portals to it and see if it can be modified to work. IF so then I might find a way to back engineer the old files.

Here's the big problem for me. I could care less about the windowed houses. I really want to get my bunker back, and maybe see some of the other structures that were added after I left the game. But there's no old POB file for the bunker to compare with so I'd be flying blind trying to make my own POB from scratch. so I HAVE to figure out how to convert the NEW POB's to work...

Sorry for the ramble but it's helping get my thoughts in order.

-Duff
 

Uli

Moderator
Staff member
Moderator
Aug 30, 2010
208
1
18
#5
When you realise what the issue is and how easy it is to fix you're going to probably hate yourself.
 
Sep 13, 2013
188
2
18
#6
Maybe, but here's what I've done, and here's my results so far. (starting from scratch)

Rename 0004DATA to 0003DATA
Rename all the PRTL0005's to PRTL0004's.

I changed all the 0000verts in the first section to just PRTL's added the 04 00 00 00 long to the beginning and reversed the order of the x,y,z points for each entry other than the first. The first PRTSPRTL I had to take the first entry (3 longs) and move it to the last, but otherwise it's not in reverse order according to my observations (you still have to add the 04 00 00 00 long to the beginning). Then I deleted all the forms & left over indx files.

Then I went through each of the each PRTL0004 data nodes and removed the leading byte as the old file didn't have it.

Finally I had to replace the door strings with the old house door strings as I was getting a funky flower as the door object LOL... I haven't tracked that down yet...

so...

At this point I have a working house mostly. The interrior is working fine other than the hue_door being something other than a door (fixed by using door_s01). The windows look out onto the world and it's neat. The doors are working, collision seems to be fine, textures appear to be fine.
However, now the exterior doesn't have clear windows anymore, they're back to the old style textures...

At this point I'm done with the small naboo house and windows. If Uli's trick is simpler then I'd be way interested in hearing it, But since I can now convert the POB to have working interriors, I'm not concerned with windows or working exteriors. The bunker house, nor any of the other non-windowed structures from the end game dont' need that anyway. :)

So next project is the bunker. I think I can do it now. wish me luck!

-Duff
 

Uli

Moderator
Staff member
Moderator
Aug 30, 2010
208
1
18
#7
The door stuff is contained in datatables/appearance/door_style.iff

and here is a tre explorer screenshot of a fixed POB file:
http://i.imgur.com/EVGZ1A5.png
 
Sep 13, 2013
188
2
18
#8
:) I'll let you know how the musty house goes after I finish a different sort of hack job in my wood shop. Damn employeer moved me into an office where the desk is built into the wall opposite the door, hence my back's to the door and I can't STAND that... So I'm improvising a solution out of 3/4" oak ply (the good stuff). Looks great with one coat of stain, but need to apply another to darken it up a bit. Back to the shop, I'll let you know about the musty thing tomorrow hopefully. I'll have time between poly-coats.

-Duff
 
Sep 13, 2013
188
2
18
#9
meh, the musty bunker is a bust. I got it working, but the collision files aren't working and I just fall through the floor before I get to the elevator, then client crash. so while my POB edits appear to be sucessfull, i'm now left with trying to figure out why I fall through the floor when I enter the cell.

What's odd is that the bunker uses the ply_all_house_lg_s03_fp1_*.flr files. Every other house I've looked into bringing into the world has it's own named .flr files. Maybe the TOC I'm using doesn't have the musty bunker flr files, or maybe it always used these generic file names... *shrug* either way i'm not certain I have the energy and desire to actually solve this one.

I'll let you know if I do decide to tackle it. I'll also try to update my proceedure above so others can use this POB edit method to get their houses in game.

-Duff
 

Timbab

Administrator
Staff member
Administrator
Moderator
Oct 6, 2010
1,044
38
48
28
Magna Germania
#10
I'd like to help, but I have zero experience with any of this stuff.

I think some of my NGE .tre files are corrupt, too, can't pull out some files/see others. Bunker being one of them.
 
Sep 13, 2013
188
2
18
#11
naw it's kewl. I'm starting to understand the file structures better but I still don't understand what Uli and yourself understand, which is what the client does with them or how they're used. If Uli's POB fix is much simpler than my own, then it's because he understands how the client is reading / interpreting the files, and I have yet to lean that. :)

right now my knowledge is such that 0008's crash the client, 0007's work. I have no idea why. LOL. Maybe if I could decompile the client I might, and that might be a good idea to try. But you guys didn't learn it overnight and I won't either. I've only been working on this stuff since early October anyway. :)

-Duff
 

Timbab

Administrator
Staff member
Administrator
Moderator
Oct 6, 2010
1,044
38
48
28
Magna Germania
#12
Well as far as I understand it, Uli still knows way more than I do, if the client can't read something based on how it was written to read something, it throws an error.

0008 for example would be a NGE file/chunk version, while 0007 is a version they used for PreCU times. So when tie client reads the file, it expects to read a 0007, but doesn't see it, because it's obviously changed and doesn't know what to do with a 0008, even if everything inside the file might be the same besides that.

The NGE client can read both the previous ones and the 'new' 0008 chunk, but if you'd use like 0010, it wouldn't know what to do with that either, because it's not written to read a 0010 and doesn't know what to do with it.

The same applies for everything, which is why the chunk/form length is a key element to a lot of the manual hex editing too, because the client will read X number of bytes based on the given chunk size number, if it's less or more, it'll stop reading at that point that's given, and try to grab what the next chunk/form is and it's size, but obviously if you cut it too late or too early, it fucks up the whole byte 'system' up from where it was 'cut' onwards.

Really simple example:

If it's FORM with the size 3 and then the client would read the 3 bytes after the FORM + 4 bytes for the size, so as a visual example, I'll mark the form/chunk size in bold:

In ASCII: FORM...abc
In HEX: 46 4F 52 4D 00 00 00 03 61 62 63

So with that, the client would read, it's a FORM, then the size as a reversed integer (3) and then the ABC, so 61 = a, 62 = b, 63 = c

But if you'd now do it like:
In ASCII: FORM...abcd
In HEX: 46 4F 52 4D 00 00 00 03 61 62 63 64

It would still only read the first 3 bytes after the FORM length, resulting in reading just 'abc', so you'd have to adjust the 00 00 00 03 to a 00 00 00 04, etc

This is just form length, not related to your problem, but it's a critical aspect of how files are read. If you' just modify existing bytes and don't add any to them, you won't stumble into problems, but the moment you change like: sample/123.wav to sample/1234.wav, you'll run into problems, because without changing the form/chunk size, it now reads sample/1234.wa

treExplorer automatically calculates the length when saving, which is why editing it there usually works, but I mostly edit in hex editors now.


Now, based on the actual file that you're having problems with, look at what it reads and if the NGE file reads it in the same way, extensions, etc, see if the way the file is read is different in any way, even if the structure is the same, if there are extra values which it can't read, etc.

Think of the client as a rulebook and if something isn't following the rules, then it can't and won't continue, until that problem is fixed. Comparing and studying working files and them comparing usually gets you the answer of why something isn't being read. I stumble over that problem constantly when I'm in the process of writing tools for each file, especially if there is like 1-2 'special' files that my program then can't read, because they're different, but I'm a total nab at programming as it is, so... :(
 
Sep 13, 2013
188
2
18
#13
makes me wonder if Uli's solution isn't the chaning of the .tre files, but in using the ProjectSWG client instead... Is there even a chance that would work with SWGEMU?
 
Sep 13, 2013
188
2
18
#15
meh, I meat the NGE client, not necessarily projectswg in particular. from his "working POB" screen shot it appears he doesn't need to change the forms within the POB file, based on your previous post, that would mean his client can read the new forms. so using a newer client maybe? *shrug*

Either way I'm taking a break from it for a while. Little frustrated about things all around, it will pass but it's best to take a break and hit it again in a few days with a clear head. :)

I just wish I could play the game again and enjoy it. Now days all I see are unrealized possibilities that I have the power to change but no time or not quite enough experience yet. *smile*

-Duff
 

Uli

Moderator
Staff member
Moderator
Aug 30, 2010
208
1
18
#16
duffstone said:
meh, I meat the NGE client, not necessarily projectswg in particular. from his "working POB" screen shot it appears he doesn't need to change the forms within the POB file, based on your previous post, that would mean his client can read the new forms. so using a newer client maybe? *shrug*

Either way I'm taking a break from it for a while. Little frustrated about things all around, it will pass but it's best to take a break and hit it again in a few days with a clear head. :)

I just wish I could play the game again and enjoy it. Now days all I see are unrealized possibilities that I have the power to change but no time or not quite enough experience yet. *smile*

-Duff
Or just throwing this wild theory out there which I have no idea where I got it from, the client can read those "newer" nodes right off the bat. Same theory can be applied to PreCu TRN and reading the tga heightmaps.

Only very few "new nodes" can't be read.
 
Sep 13, 2013
188
2
18
#17
Uli said:
duffstone said:
meh, I meat the NGE client, not necessarily projectswg in particular. from his "working POB" screen shot it appears he doesn't need to change the forms within the POB file, based on your previous post, that would mean his client can read the new forms. so using a newer client maybe? *shrug*

Either way I'm taking a break from it for a while. Little frustrated about things all around, it will pass but it's best to take a break and hit it again in a few days with a clear head. :)

I just wish I could play the game again and enjoy it. Now days all I see are unrealized possibilities that I have the power to change but no time or not quite enough experience yet. *smile*

-Duff
Or just throwing this wild theory out there which I have no idea where I got it from, the client can read those "newer" nodes right off the bat. Same theory can be applied to PreCu TRN and reading the tga heightmaps.

Only very few "new nodes" can't be read.
fair enough, but I know better than to ask how. Hopefully I have the ability to figure it out eventually, but for now like I said I'm gonna shelf it until I can get a better perspective.

-Duff
 
Sep 13, 2013
188
2
18
#18
Uli,
Not sure if you can (or will) answer this, but what does the CRC field do in the POB? I've been plugging lots of values into the calculator but i'm not finding any matches. is it some sort of parity for the entire file or something? is it a pointer for something?
 

Timbab

Administrator
Staff member
Administrator
Moderator
Oct 6, 2010
1,044
38
48
28
Magna Germania
#19
Uli said:
Or just throwing this wild theory out there which I have no idea where I got it from, the client can read those "newer" nodes right off the bat. Same theory can be applied to PreCu TRN and reading the tga heightmaps.

Only very few "new nodes" can't be read.
My bad then.

Logic still stands for other stuff though.
 

Uli

Moderator
Staff member
Moderator
Aug 30, 2010
208
1
18
#20
duffstone said:
Uli,
Not sure if you can (or will) answer this, but what does the CRC field do in the POB? I've been plugging lots of values into the calculator but i'm not finding any matches. is it some sort of parity for the entire file or something? is it a pointer for something?
Was the crc for something can't remember exactly, possibly the object path iff or the pob itself.