Wrp File Format - OPRWv17 to 24: Difference between revisions

From Bohemia Interactive Community
Jump to navigation Jump to search
mNo edit summary
m (cleaned up discussion)
Line 180: Line 180:




----
----
----


Are you still interested in some input/observations or are you fine now? -- [[User:Sy|Sy]] 09:16, 9 October 2008 (CEST)
'''Discussion'''
: Of course I am still interested. Gimme all you have and know ;) --[[User:T D|T_D]] 15:18, 9 October 2008 (CEST)


----
'''TextureGridModels'''
Ok, first let's talk about the area of the file you call the 'Road Block'.
As you've noted it appears at first glance to be road models with intermittent x00 blocks.
First of all find the first lot of road models in your file and you'll see there is a 'block' of x00's before these... Navigate backwards through these x00's till they stop being x00's.
This is the ''real'' start of this area of the file.
You should see 3 int's before it starts into the x00's.
:The first int is undecided about at the moment.
:The second int is the number of 'classless' objects there are in the file. Further talk about this later as there is a small ''issue'' with this.
:The third int would be better described as a long because it's the total size of the structure of this area of the file. This, structure size int is then the starting point for this area of the file.
At the moment a preference exists to describe this part of the file as the 'TextureGridModels' area.
This is because it is a area that is of the form int[TxGrid_Z, TxGrid_X].
Although the models defined within this area always seem to be of the 'road' type I have for the moment decided to call the area (as mentioned) 'TextureGridModels' to identify this area as a list of models within each cell of the 'TextureGrid'.
It is imagined at some later date this area would most likely be renamed to something like 'TextureGridRoads' or 'RoadNetworks' or just 'Roads' or some such.
To illustrate... If there were NO roads and the texture grid was 128 x 128 then this entire block would be 128 x 128 0x0000 entries, or 65,536 bytes in length. This size would also be equal to the 'Third' integer I mentioned above.
Everytime you come across a number in one of these 'slots' (if you will) it represents how many 'sub-objects' are present in this one cell.
Then of course the 'sub-objects' are defined. At the end of these 'sub-objects' you are into the next 'cell' of the grid array.
Do you understand this?
----
'''Objects'''
As you've already indicated the structure that directly follows the above 'TextureGridModels' area is of course the 'Objects' list. Probably, more accurately it's the list of 'Classless' models that are present on the island.
The problem is that the '2nd int' that was mentioned at the beginning of the 'TextureGridModels' area it apprears now is the 'maximum' object Id that is present on the island. This, is because in the are often quite a number of items less than this, but never more, of course.
Eg. In a  large island that comes with the 'ca' mod the NoOfObjects or MaxObjectId is 637,876 and the number of actual objects in the list array is 602,481. And, the highest Object Id in this list array is 637,876.
It's unfortunate at this time that the array does not contain 637,876 entries, with some of them say having an Object Id of -1 or something as this would then mean that the '2nd int' truly would indicate the NoOfObjects in the array list of 'Classless Objects'.
Currently, this makes iterating this area a little difficult.
----
'''Map Object Features'''
Dunno, if this is an appropriate name for this area but you have an accurate description for it anyway.
The only thing to add is that it extends to the the EndOfFile. And, there are a number of other values + another type.
Some quick psdeuo-code...
<code><nowiki>
while (filepointer < filesize)
{
  //type1 = Id, x, z
  //type2 = Id, Bounds (float[8])
  //type3 = int[2], float[4]
  //type4 = Id, Bounds (float[8]), Colour (byte[4])
  //type5 = Id, float[4]
  type=readint();
  switch(type)
  {
    case 0:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 1:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 2:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 3:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 4:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 8:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 9:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 10:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 11:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 14:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 15:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 16:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 17:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 18:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 20:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 22:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 23:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 24:
        {
            typeid = TypeIdEnum.Type2;
            break;
        }
    case 25:
        {
            typeid = TypeIdEnum.Type3;
            break;
        }
    case 26:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 27:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 28:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 29:
        {
            typeid = TypeIdEnum.Type4;
            break;
        }
    case 30:
        {
            typeid = TypeIdEnum.Type1;
            break;
        }
    case 31:
        {
            typeid = TypeIdEnum.Type2;
            break;
        }
    case 32:
        {
            typeid = TypeIdEnum.Type2;
            break;
        }
    case 33:
        {
            typeid = TypeIdEnum.Type3;
            break;
        }
    case 34:
        {
            typeid = TypeIdEnum.Type5;
            break;
        }
    default:
        {
            typeid = TypeIdEnum.Nothing;
            break;
        }
  }
}
</nowiki></code>
----
ummm i dunno what the above table (now) achieves. it is missing two id's (13 and 19) and is not as informative as the previous shorter version lacking floats and longs as part of it's info
Mikero
[[User:Mikero|Mikero (nee Ook?)]]
:This table existed before the one above. I added all bits that were found out at the top. Of course this table isn't needed anymore and if it confuses you too much you are free to delete it ;) --[[User:T D|T_D]] 14:02, 4 December 2008 (CET)
----
'''TextureIndices / TextureGridMaterials'''
This area as you are probably aware maps the 'TextureGrid' to the material files and (near the beginning of the file) is almost analysed.
----
That's all to add at this stage. Besides, what you had originally was enough to get a working .pew out of...
If the style in which the above info. is conveyed maybe difficult to envisage or follow, apologies.
Was this at all helpful? If you have any questions or further insights...
: thank you very very much for this info. I guess I understood most of your descriptions and very appreciate your effort. I will integrate your infos in the next time (my holidays ended yesterday, so I dont have as much time as I had the last months).
:For the TextureIndices block I found out that it is organized in 8x4 blocks. If the block starts with 0x00 you just read in 32 short values describing the block. Otherwise you mostly see 2 identical short values describing one 8x4 block filled just with this value. Sometimes instead of the 0x00 value or the 2 identical short values, other values occur probably describing what sort of data follows and what position the next 8x4 block in the whole ''texX'' x ''texZ'' block has. But I am still not sure what order these 8x4 blocks follow. The TextureIndices block also seems to start with some "header" data which I couldnt identify so far. --[[User:T D|T_D]] 12:11, 13 October 2008 (CEST)
:For the TextureIndices block I found out that it is organized in 8x4 blocks. If the block starts with 0x00 you just read in 32 short values describing the block. Otherwise you mostly see 2 identical short values describing one 8x4 block filled just with this value. Sometimes instead of the 0x00 value or the 2 identical short values, other values occur probably describing what sort of data follows and what position the next 8x4 block in the whole ''texX'' x ''texZ'' block has. But I am still not sure what order these 8x4 blocks follow. The TextureIndices block also seems to start with some "header" data which I couldnt identify so far. --[[User:T D|T_D]] 12:11, 13 October 2008 (CEST)
::Ah you seem to already know about the TextureIndices very well. Good to know ^^
:Err, question: if the objectblock only contains the classless objects, where are the objects with classes then? In model list? --[[User:T D|T_D]] 23:52, 13 October 2008 (CEST)
::On closer inspection of the 'Classed Models' - while they are indeed classed, that is they have a 'Land_' config definition each model entry in this array is also present in the 'Classless Models' array.
::This then means that the 'Classless Models' array would be better referred to as just the 'Models' array... in a .pew these would be the model templates.
::As a test create a blank island and setup just 1 config'd/classed/Land_ house model as a template & 1 Tree template.
::Place the house model 20 times in different locations. Place 50 Trees in different locations.
::You'll see these 20 different located houses are in the 'Classed Models' array. Same model, 20 different locations.
::Then in the 'Models' array there will be 2 'Models' / 'Model Templates'. 1 for the House & 1 for the Tree.
::Then in the Objects or Model Instances array you'll have 20 House instances & 50 Tree instances. The 20 house instances transforms x,y,z's will exactly match the x,y,z's from the 'Named Models' array.
:::It seems that BIS has quite some redundant data then:
:::#like you said the positions of "classed" models in objectslist and "classed" models block
:::#road model pathes in model list and in TextureGridModels block (an index to the roadpath would have been enough in my opinion)
:::#transform matrix in Objectlist and in roadblocks
:::#"classed" modelnames also present in modellist
:::Dunno what they want to achieve with that --[[User:T D|T_D]] 15:27, 14 October 2008 (CEST)
:::One can infer quite alot about the capabilities of certain things based upon the data structures used. Sometimes redundancy seems the case but it sort of is and sort of isn't. Alot of data structures are this way to allow direct injection into shader code or directx structures with little or no effort. So, sometimes some of the structures on the surface look inefficient but when used in DirectX can save alot of pre-precessing. Keep-it-Simple-Stupid could be a good term to use. Although, saying this there are some quite redundant data structures lying about.
:::The .paa file format is fifo for instance which is ideal for streaming...
:::Maybe something like...
:::<code><nowiki>
structRoadNetwork[TxGridZ,TxGridX] TextureGridRoads;
structRoadNetwork
{
  int NoOfRoadParts; // Zero or More...
  if (NoOfRoadParts != 0)
  {
      structRoadPart[NoOfRoadParts] RoadParts;
  }
};
structRoadPart
{
  short NoOfRoadPositions;
  structRoadPosition[NoOfRoadPositions] RoadPositions;
  int Flags;
  asciiz Model;
  float[4,3] Transform;
};


structPoadPosition
{
  float X;
  float Y;
  float Z;
};
</nowiki></code>


:::A T-Juntion will have 4 road positions StartPos1, EndPos1, StartPos2, EndPos2... :)
:The unknown int in the the TextureGridRoads block is the checksum of the packed data before so it does not belong to the TextureGridRoads block --[[User:T D|T_D]] 14:54, 17 October 2008 (CEST)


::::The unknown int in the the TextureGridRoads block is the checksum of the packed data before so it does not belong to the TextureGridRoads block --[[User:T D|T_D]] 14:54, 17 October 2008 (CEST)
::Agreed, yeah... saw that, also, maxObjectID isn't part of the TextureGridRoads structure and it will probably end up really being NoOfObjects.


::::Agreed, yeah... saw that, also, maxObjectID isn't part of the TextureGridRoads structure and it will probably end up really being NoOfObjects.
::No it is not NoOfObjects. Already checked that by reading objects from 8WVR and comparing readed objects with this value --[[User:T D|T_D]] 04:33, 18 October 2008 (CEST)


::::No it is not NoOfObjects. Already checked that by reading objects from 8WVR and comparing readed objects with this value --[[User:T D|T_D]] 04:33, 18 October 2008 (CEST)
::Experience has taught me that one can never be to certain about these things until one is certain, if you know what I mean... :)
::It maybe MaxObjectId it maybe not... time will tell.


::::Experience has taught me that one can never be to certain about these things until one is certain, if you know what I mean... :)
::Building the standard SampleMap.pew into a 8wvr.wrp yeilds 5134 obejcts, then into a oprw18.wrp yeilds 5132 in the MaxOjectId/NoOfObjects field. If you add 1 to this value this is the number of obejcts that happens to be in the array of objects in this particular file.
::::It maybe MaxObjectId it maybe not... time will tell.
::However, this senario doesn't hold true for a certain 'ca' mod oprw18.wrp. That value (as I mentioned earlier) is 637,876 and the number of actual objects in the list array is 602,481.
::This may be because there is a prior structure in the file that denotes out of the 637,876 objects ones that are not specified in the Objects array... dunno, well see if that pans out.
::The most logical senario is that this is the NoOfObjects less some other structure count = ObejctList count.
::If this isn't the senario then there maybe another ObjectCount item lying around the file somewhere that does indicate how many objects are in the objects list array.


::::Building the standard SampleMap.pew into a 8wvr.wrp yeilds 5134 obejcts, then into a oprw18.wrp yeilds 5132 in the MaxOjectId/NoOfObjects field. If you add 1 to this value this is the number of obejcts that happens to be in the array of objects in this particular file.
::::However, this senario doesn't hold true for a certain 'ca' mod oprw18.wrp. That value (as I mentioned earlier) is 637,876 and the number of actual objects in the list array is 602,481.
::::This may be because there is a prior structure in the file that denotes out of the 637,876 objects ones that are not specified in the Objects array... dunno, well see if that pans out.
::::The most logical senario is that this is the NoOfObjects less some other structure count = ObejctList count.
::::If this isn't the senario then there maybe another ObjectCount item lying around the file somewhere that does indicate how many objects are in the objects list array.


:: this block ''ushort[gridZ][gridX] unknown //packed'' seems to consist of random numbers. So my guess is, that these random values are used to calculate the positions of the clutter models, so that the island is always looking the same without saving the pos of every grass model. Could this be the case? --[[User:T D|T_D]] 13:50, 19 October 2008 (CEST)
:this block ''ushort[gridZ][gridX] unknown //packed'' seems to consist of random numbers. So my guess is, that these random values are used to calculate the positions of the clutter models, so that the island is always looking the same without saving the pos of every grass model. Could this be the case? --[[User:T D|T_D]] 13:50, 19 October 2008 (CEST)

Revision as of 18:51, 4 December 2008


Template:unsupported-doc

Current state of my researches on OPRW18 file format. Any additions, suggestions, etc. are appreciated.

/* Legend ** ** byte 8 bits, unsigned. Normally used for flags. ** ushort 2 bytes, unsigned. ** ulong 4 bytes, unsigned. Normally used for offsets and ID's ** float 4 bytes. signed. The preferred method of storing negative values (such as elevation) */ char[4] sig //"OPWR" ulong version //0x12 = 18 ulong texX,texZ //textures cell dimensions. 256x256 eg ulong gridX,gridZ //elevations cell dimensions. 256x256 eg float gridSize //in meters (normally 50.0) //still unknown stuff here //followed by Peaks { ulong nEntries float XZY[nEntries][3] //Position of the Peak (Z is height) } TextureGridMaterials[unknown size] //see discussion below. Not fully defined (yet) ushort PackedRandom[texZ][texX] // probably random values needed for calculating clutter models position byte PackedUnknown[gridZ][gridX] // seems to be related to clutters or sat mask distance as it is responsible for the clutterbug float PackedElevation[gridZ][gridX]// /* packed data corresponds to the standard pbo compression format */ Rvmats { ulong nRvmatEntries //Number of Rvmat entries (at least one) RvmatEntry { Concatenated Asciiz path strings }[nRvmatEntries] } /* Note 1) there is always a minimum of one rvmat entry. The first entry is *always* 0x00 0x00 (no file) the *actual* number of files listed is therefore *always* one less than nRvmatEntries 2) BI frequently use concatenated strings. One Asciiz string, followed by zero or more others. in this case, there is always only one filename listed and thus the above could be viewed as ConcatenatedString { asciiz rvmat_path byte 0x00 } */ } Models { ulong nModels //number of models asciiz model_path[nModels] //e.g. ca\buildings\kostelik.p3d } /* ** The ModelIndex in the ObjectsBlock below is a direct reference to one of the above p3d files for that object entry. If there are 99 nModels then ** the Object entries will contain ModelIndexes between 0 and 98 */ Classed Models { ulong nClassedModels //number of classed models Model { asciiz class_name //e.g. Land_Hangar asciiz model_path //e.g. ca\buildings\kostelik.p3d float pos[3] //XZY ulong unknown }[nClassedModels] } //something grid related[size unknown] // followed by: byte[texZ][texX] bytes2 //seems to be connected to roads, runways and special grounds byte[gridZ][gridX] bytes3 //unknown but 0x00 for forests and roads and sea, mostly 0x03 and rarely 0x02 ulong maxObjectID //not number of objects ulong ObjectBlockOffset //amount of bytes used for RoadNets TextureGridRoads { RoadNet[texZ,texX] RoadNets; //probably organized in texture grid cells to realize terrain streaming RoadNet { ulong NoOfRoadParts // Zero or More... RoadPart[NoOfRoadParts] RoadParts } RoadPart { ushort NoOfRoadPositions float[NoOfRoadPositions][3] RoadPositions //XYZ byte[4] Flags asciiz Model float[4,3] matrix } } /* Because of the variable length asciiz strings, the actual position of the following Objects block cannot be pre-calculated. Therefore, ** the ObjectBlockOffset is also the size (in bytes) of the Roads Block ** ** On a 256 x 256 cell island: Each cell *unconditionally* contains a roadnet entry. If there are no road(s) in that cell, there are no roadparts. There could, alternatively be ** multiple roads in that cell, and, multiple parts to each road. ** ** For a 256 x 256 cell island with NO roads, the size of this block, the ObjectBlockOffset , would be 256 x 256 x 4 bytes. */ Objects { Object { ulong ObjectID ulong modelIndex float[12] matrix ulong 0x02 ?? } } /* ** modelIndex is a direct reference to the Nth model entry in the Model Block above. If there are 99 p3d files listed in the Model Block, then any given Object entry will contain a ** modelIndex between 0 and 98. ** ** For any island, there are more than 100,000 (p3d) objects on the terrain. The ObjectID uniquely identifies *that* tree, at *that* position. The *same* tree, house, bush, will be ** used many times on the island at different locations. Thus many object entries contain the same modelIndex. Eg, the same, p3d tree. */ MapInfo { ulong type if(type in [0,1,2,10,11,13,14,15,16,17,22,23,26,27,30]) type1 if(type in [24,31,32]) type2 if(type in [25,33]) type3 if(type in [3,4,8,9,18,19,20,28,29]) type4 if(type in [34]) type5 //type1 //{ // ulong ObjectId // float x, z //} //type2 //{ // ulong ObjectId // float[4][2] Bounds //} //type3 //{ // long[2] unknown // float[2][2] line? //} //type4 //{ // ulong ObjectId // float[4][2] Bounds // byte[4] Color //rgba //} //type5 //{ // ulong ObjectId // float[2][2] line? //} }


Discussion

For the TextureIndices block I found out that it is organized in 8x4 blocks. If the block starts with 0x00 you just read in 32 short values describing the block. Otherwise you mostly see 2 identical short values describing one 8x4 block filled just with this value. Sometimes instead of the 0x00 value or the 2 identical short values, other values occur probably describing what sort of data follows and what position the next 8x4 block in the whole texX x texZ block has. But I am still not sure what order these 8x4 blocks follow. The TextureIndices block also seems to start with some "header" data which I couldnt identify so far. --T_D 12:11, 13 October 2008 (CEST)


The unknown int in the the TextureGridRoads block is the checksum of the packed data before so it does not belong to the TextureGridRoads block --T_D 14:54, 17 October 2008 (CEST)
Agreed, yeah... saw that, also, maxObjectID isn't part of the TextureGridRoads structure and it will probably end up really being NoOfObjects.
No it is not NoOfObjects. Already checked that by reading objects from 8WVR and comparing readed objects with this value --T_D 04:33, 18 October 2008 (CEST)
Experience has taught me that one can never be to certain about these things until one is certain, if you know what I mean... :)
It maybe MaxObjectId it maybe not... time will tell.
Building the standard SampleMap.pew into a 8wvr.wrp yeilds 5134 obejcts, then into a oprw18.wrp yeilds 5132 in the MaxOjectId/NoOfObjects field. If you add 1 to this value this is the number of obejcts that happens to be in the array of objects in this particular file.
However, this senario doesn't hold true for a certain 'ca' mod oprw18.wrp. That value (as I mentioned earlier) is 637,876 and the number of actual objects in the list array is 602,481.
This may be because there is a prior structure in the file that denotes out of the 637,876 objects ones that are not specified in the Objects array... dunno, well see if that pans out.
The most logical senario is that this is the NoOfObjects less some other structure count = ObejctList count.
If this isn't the senario then there maybe another ObjectCount item lying around the file somewhere that does indicate how many objects are in the objects list array.


this block ushort[gridZ][gridX] unknown //packed seems to consist of random numbers. So my guess is, that these random values are used to calculate the positions of the clutter models, so that the island is always looking the same without saving the pos of every grass model. Could this be the case? --T_D 13:50, 19 October 2008 (CEST)
Retrieved from "https://community.bistudio.com/wiki?title=Wrp_File_Format_-_OPRWv17_to_24&oldid=43838"