Wishful 3D file format

cheetah3d
Unregistered
 
Post: #61
I have just one wish. Please go binary. I made some very bad experiences with text file formats. Cry
Binary file formats are definitly easier, much faster and robuster to read, and smaller in size of course. The only thing which speaks for ASCII is that it is human readable. But if you conssider that nobody will read 10.000 vertices it isn't a good pro argument.
And making both ASCII and Bianry is also quite painful because you always have to write and maintain two loaders and two exporters for the same context. Not that funny if you've already written a dozen of loaders and exporters.
Binary rules Love
Quote this message in a reply
⌘-R in Chief
Posts: 1,265
Joined: 2002.05
Post: #62
Uhhh..... I'm not sure I follow. Wouldn't it be exactly where it says? glTranslatef doesn't draw anything, obviously, and you certainly wouldn't translate, do a glVertex3f(0, 0, 0), move back, translate for the next vertex etc., etc.. You translate to where you want the center of your model to be, and each vertex would map directly to a glVertex call. Or am I really misunderstanding what you're asking?
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #63
PowerMacX Wrote:From the specs:
I agree that the format should be extensible, but "making no assumptions" about animations doesn't sound that good. If this format should become widely used, it has to have at least one standard for key framed animation and one for bones, otherwise general exporters & importers for animations would be impossible.
Yes, there should (and are and will) be some standard ways to do different types of animation, but there should also be room for custom application-specific data. The idea here is that extra data won't break apps that don't understand it and that different apps can use the same data in different ways - if they choose to do so. The load/save routines actually make no assumptions about _any_ chunk, it's the routines built on top of the generic I/O that decide what to do with each chunk (which can include simply ignoring it).

Here's a practical example: Lets say you're using someone else's closed-source editor that imports a bunch OBJ or 3DS frames and turns them into a keyframed E3M file. Then you decide that your app needs to support some sort of tag or hook object to link multiple models (ala Quake's MD3). You can write your own tool based on the generic I/O to add these hooks, yet still go back to that closed-source app and make changes to your models without loosing your custom data.
Quote this message in a reply
Moderator
Posts: 3,579
Joined: 2003.06
Post: #64
Yeah, so you mean making the format even be a "wrapper" for other formats, so to speak. IOW, it shouldn't be out of whack to contain, say, a .obj or .md2 within the structure of the file along with the other custom chunks (or data blocks as I call them). I really like the tag system in md3. I think it's a little awkward to call them "tags" though. I like your "hook" terminology better to describe other similar external files. The system makes a lot of sense in that it's a great way to reference other files to fit within the structure of the model being described.
Quote this message in a reply
Member
Posts: 446
Joined: 2002.09
Post: #65
AnotherJake Wrote:Yeah, so you mean making the format even be a "wrapper" for other formats, so to speak. IOW, it shouldn't be out of whack to contain, say, a .obj or .md2 within the structure of the file along with the other custom chunks...
That can be done if needed, though I'm not sure it would be all that useful to embed another 3D format in it's entirety. I have used PNG data blocks to embed images in some cases, though I prefer keeping textures/shaders separate for games.
Quote this message in a reply
Moderator
Posts: 3,579
Joined: 2003.06
Post: #66
I actually meant *referencing* to an external file, I could have been more specific. I think it's a bad idea to embed other files or data like textures, etc.
Quote this message in a reply
Member
Posts: 144
Joined: 2004.07
Post: #67
I've recently been building up more and more interest in completely rewritting the currrent model format I'm using (for a format the community thinks is more friendly and can benefit from all our work) and think it'd be cool to _possibly_ start nailing out some initial features and start developing the format (I'd work on a C4D plugin for it).

One thing I'd like to add to the dicussion is maybe getting a little help from the guy making animadead to create a powerful cooperative environment.
Quote this message in a reply
DoG
Moderator
Posts: 869
Joined: 2003.01
Post: #68
... [/hide] so many projects so little time [hide] ...
Quote this message in a reply
Member
Posts: 153
Joined: 2004.12
Post: #69
I know that this would come into play MUCH later but... It would be really cool if the format could handle embedded materials. I just started working with FBX files and im really digging the fact that i can do a lot of testing and 'setup' without having to deal with getting arbitrary textures, just to distinguish between multiple objects in my scene.

There was a long silence...
'I claim them all,' said the Savage at last.
Quote this message in a reply
kberg
Unregistered
 
Post: #70
I've been giving this some thought over the last while, and I think this might not be the best way to approach the problem...

Once a mesh is loaded into memory, the file format becomes almost irrelevant to a developer. In fact, a new file format can only offer the following features above and beyond current formats:
- better compression, and therefore smaller memory footprint on the HD
- more included resources or meta-info about a mesh beyond simply vertices and connectivity data; such as materials, textures, joints, keyframe data, scene info (lights?), baked lighting, etc..

The second issue is addressed by simply choosing an appropriate format to begin with (and specific formats exist for pretty much any use I can think of!)

Almost all other issues such as loading speed, ease of use (API), shadows, collision detection, etc, are properties of the code used to load and manipulate the data. While a new all-inclusive file format would certainly be great, I think as a first step it would be far more productive to produce a well thought out, extremely easy to use, easily extensible, and very consistent library capable of loading a multitude of 3D file formats, along with a set of fast and easy to use algorithms to operate on the data once loaded.

I would start by identifying the types of 3D data files to include, and then pick at least one format of each type to write a parser for. Types would include:
Static meshes: wrl, .obj, .lwo, .3ds
Keyframed or morphing meshes: .md2, .md3, .3ds
Skinned or bone based meshes: .md5
Flat terrains: heightmap,
Spherical terrains (planets): heightmap
Map?, Level?: this probably can be left out or broken down, but quake style bsp maps, etc..

I would then create custom classes for each type of 3D data, with sets of algorithms, some common to all and some specific to the data type they are operating on. For example, an update_lod() call could call a mesh decimation algorithm for a static, keyframed, or skinned mesh, while it may be the update algorithm for a flat terrain based on the ROAM algorithm, etc.. Dynamic mesh types such as skinned or frame based should share identical function names where possible.

The set of common algorithms could include rendering in a variety of states (flat, wireframe, etc), level of detail, collision detection (either custom built or through the abstracted use of a physics library), shadow volume creation, ray intersection, etc..

We could continue adding to this list as iDGers require certain features for their games, and end up with an extremely robust and full featured library. Something like this may or may not already exist; if it does we could focus on porting it if necessary and then writing a set of simple, easy to follow tutorials.

Yes this is a LOT of work, but once we've hammered out a full-featured library we would be in a much better position to decide what should be included in a new file format, and what the best way to implement that format would be.
Quote this message in a reply
Member
Posts: 153
Joined: 2004.12
Post: #71
kberg's idea makes a whole lot of sense.

The FBX file format attempts to do pretty much what where talking about... and it sorta does. It covers a lot of ground with a lot of flexibility. The side effect however is an extremely messy, verbose, and overall, pretty shitty format. It does a lot of things, but none of them very well.

Im sure we could create one format that covers everything, but im afraid it'll suffer from the same problems FBX does. Perhaps we should accept the fact that there is no one format that fits all our needs, and streamline the parsing process so its not such a pain, to use the correct tool (format) for the job.

There was a long silence...
'I claim them all,' said the Savage at last.
Quote this message in a reply
DoG
Moderator
Posts: 869
Joined: 2003.01
Post: #72
While kberg has some valid points, it seems like he didn't read the start of this thread. The goal is to be flexible, and have a file format of which any given game/app only has to support some minimal subset, such as the basic scenegraph, and can ignore the rest of the file. Some form of tagged file format seems best for this, it being ASCII or binary doesn't make a difference.

And, yes, a library/parser should also be available to get started with.
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #73
what, like 3ds?

there are plenty of good file formats out there, which do as much or as little as possible. what doesn't exist is a good library to load them. that's what the community needs. ivory tower file format designs are a fine academic exercise, but as hangt5 says, you'll just end up with FBX, and nobody will use it unless you give them a nice library and some nice export/conversion tools. if you give them a nice library, they won't need to ignore bits of the file format.

all this i've said before......

if this is indeed an ivory tower exercise, perhaps it should be solidly branded as such. if it's intended to be useful, then kberg's comments are entirely relevant. let's not talk at cross purposes.
Quote this message in a reply
johnb003
Unregistered
 
Post: #74
Lightbringer: I am the author of animadead, and I would be very much interested in hearing your suggestions. I think animadead is right on target for this wishlist.
It is currently written in a text format but in previous versions I've used a binary format and is very easy to create. I chose the text format for readability.

The whole point of the library is in fact skeletal animation.

It doesn't do keyframe animation per-se but it does export each frame of an animation and interpolate between those frames which in a sense IS keyframe animation.

The format itself doesn't have extra user information at the moment, but when textures are read from the file, it calls a user defined texture loader, and can store user-defined data in the the model structures (such as the texture id).

It currently has an exporter for maya and I intend to make ones for 3ds, lightwave, and more.

The library does load the file format and even provides functions for setting up blends between different animations and provides and interface for getting the final matrices used to deform the mesh which can be used in a shader.

the meshes are stored with per-vertex texture coordinates (and geometry is optimized for use in vertex arrays) Vertices that share the same location, normal, and uv-coorinates are combined.

Materials need some work. I currently only read the defuse map (color) texture from the modeler. The rendering is actually not part of the library and that level is up to the developer, which means it can easily conform to work with the rendering design of a given engine.

I'm very interested in making improvements to the library and I could use any feedback. I wrote the library with the intent of it being as portable and cross-platform as possible. The library itself doesn't actually have any dependencies other than c++, but I did write an example program that uses sdl, which works on the mac, and opengl.
Quote this message in a reply
Moderator
Posts: 771
Joined: 2003.04
Post: #75
OneSadCookie Wrote:[...] you'll just end up with FBX, and nobody will use it unless you give them a nice library and some nice export/conversion tools. if you give them a nice library, they won't need to ignore bits of the file format.

You mean, something like this?

It can read/write FBX, and import from 3DS, DXF and OBJ.

Some notes though:
* You need to be logged in for the download to work
* Annoyingly enough, they (Alias) made an installer... and one that messes up your permissions! I made the mistake of choosing my Desktop folder as the destination folder: it didn't install everything in a folder on the Desktop as one may expect, everything got all over the place Mad and the permissions got set to "No access" Mad
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  what's the most suitable graphic file format when deadling with OpenGL? anthony 8 4,843 Feb 16, 2007 06:25 PM
Last Post: PowerMacX
  Reverse Engineering 3d File Format nabobnick 1 3,021 Aug 13, 2004 12:11 PM
Last Post: nabobnick