Little Endian to Big Endian

Jones
Unregistered
 
Post: #1
I am attempting to load a file type that is always encoded as little endian. I have all the code for opening the file type and taking the data in done, I just need to know how I should go about converting the endianess of said data. Apparently, converting it to "Host" format, can do this, but only sometimes, so that's not the greatest solution. I found this code on a forum:

Code:
inline void endian_swap(unsigned short& x)
{
    x = (x>>8) |
        (x<<8);
}

inline void endian_swap(unsigned int& x)
{
    x = (x>>24) |
        ((x<<8) & 0x00FF0000) |
        ((x>>8) & 0x0000FF00) |
        (x<<24);
}

// __int64 for MSVC, "long long" for gcc
inline void endian_swap(unsigned __int64& x)
{
    x = (x>>56) |
        ((x<<40) & 0x00FF000000000000) |
        ((x<<24) & 0x0000FF0000000000) |
        ((x<<8)  & 0x000000FF00000000) |
        ((x>>8)  & 0x00000000FF000000) |
        ((x>>24) & 0x0000000000FF0000) |
        ((x>>40) & 0x000000000000FF00) |
        (x<<56);
}

I'm not sure how it works, except that it can convert unsigned shorts and ints. Which I need, so if that works, good, but I also need to be able to read char[] data from the file. What kind of routine would I use in that case? And do I convert the data *after* it's read from the file? (Into designated structs.)

Also, does this process cause a big performance drop, usually? If it does, I'll just right a program for taking all my games data, and cycling it to big endian before hand. (Before distributing the game, for example.)

Any help/suggestions, please?

Thanks! Smile
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #2
a char is only one byte, so doesn't need its byte reordered Rasp
Quote this message in a reply
Member
Posts: 161
Joined: 2005.07
Post: #3
There are two ways to read in char arrays from what I know, although it depends on how it's being stored in the file:

1) Each "string" in the file has a short or long specifying the length of the string, then you just read in that many chars. If you need this method, make sure you byte-swap the length of the string of course!
2) If the file does not have a length specified, odds are it just loops by reading in one char at a time until it hits the null character.

Swapping endianness is very quick, so don't bother creating new data files for everything.

And finally, storing the raw data into structs depends entirely on the format of the file and your implementation of the data structures, so I can't really answer that.
Quote this message in a reply
Jones
Unregistered
 
Post: #4
Thanks for your help, OneSadCookie and mikedaman, your help has clarified things greatly. Is there any way of checking the endianess of the system by code is running on? (If I wanted to make a universal binary for PPC and x86 macs, or I wanted my code to be completely window/mac compile compatible.)

PS: Perhaps a silly question, but the 64bit-dness of some processors won't affect endianess in any way, would it?
Quote this message in a reply
Moderator
Posts: 1,140
Joined: 2005.07
Post: #5
Code:
int checkEndian()
{
   int test = 1;
   if ((*(char *)&test[3] == 1)
      return BIG_ENDIAN;
   else
      return LITTLE_ENDIAN;
}

64 bit computers are the same as 32 bit as far as endianness is concerned.
Quote this message in a reply
Jones
Unregistered
 
Post: #6
akb825 Wrote:
Code:
int checkEndian()
{
   int test = 1;
   if ((*(char *)&test[3] == 1)
      return BIG_ENDIAN;
   else
      return LITTLE_ENDIAN;
}

64 bit computers are the same as 32 bit as far as endianness is concerned.

Thanks! I actually found that same source within a couple of minutes of searching. Rasp Should've googled first.

OneSadCookie mentioned that char's are one byte, so no byte ordering problem. But what If I'm reading a set of chars of unknown length, into an array. Would the word "hello" be stored (figuratively) as "olleh" if read on the wrong endianess of machine. Then again, ever item in the array would not actually become an array until it is loaded onto the memory stack anyway, correct?

EDIT: What if I need to swap... *gasp* a float? (That decimal point must thrown in a complication... *quakes in fear*).
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #7
any array is stored with element zero first, then one, etc. Only bytes within primitives may be swapped.

you never need to determine endianness at runtime; it's always known at compile time.

Code:
#if defined(__BIG_ENDIAN__)
// something like PowerPC
#else
// something like Intel
#endif
Quote this message in a reply
Jones
Unregistered
 
Post: #8
But my code should be able to swap stuff only if It needs too.
(I'm not sure what you mean.)
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #9
I mean you should test at compile time (with the C preprocessor and #if) to see whether you need to swap stuff, not (redundantly, since you already know the answer) at run-time.

eg. if you have a little-endian data file:

Code:
#if defined(__BIG_ENDIAN__)
readMyDataFileWithByteSwapping();
#else
readMyDataFileTheEasyWay();
#endif
Quote this message in a reply
Oldtimer
Posts: 834
Joined: 2002.09
Post: #10
First off, avoid writing pure floats to file or network, since they're a bit unreliable. Instead, convert them to a fixed 16.16 format or something like it. If you insist, read this article: http://www.gamedev.net/reference/article...le2091.asp Smile
Quote this message in a reply
Jones
Unregistered
 
Post: #11
Fenris Wrote:First off, avoid writing pure floats to file or network, since they're a bit unreliable. Instead, convert them to a fixed 16.16 format or something like it. If you insist, read this article: http://www.gamedev.net/reference/article...le2091.asp Smile

Well, I'm not the one who's encoding them. Whoever wrote the format of this file type decided it would have floats in it. (Damn them... Rasp Wink )

Thanks for the link tho!
Quote this message in a reply
Jones
Unregistered
 
Post: #12
OneSadCookie Wrote:I mean you should test at compile time (with the C preprocessor and #if) to see whether you need to swap stuff, not (redundantly, since you already know the answer) at run-time.

eg. if you have a little-endian data file:

Code:
#if defined(__BIG_ENDIAN__)
readMyDataFileWithByteSwapping();
#else
readMyDataFileTheEasyWay();
#endif

If I wrote this:

Code:
static bool CPU_BIG = FALSE;

#if defined(__BIG_ENDIAN__)
    CPU_BIG = TRUE;
#endif

And then checked what CPU_BIG was every time I read, it would be ok, si? Annoyed
Quote this message in a reply
Moderator
Posts: 1,140
Joined: 2005.07
Post: #13
To swap floats, you can do a cast such as this:
*(unsigned int *)&myFloat
and then swap it as you would an unsigned int. I've had no problems with floats, since they are an IEEE standard. To answer your other question, the only time you need to swap the bytes are if each individual piece of data is > 1 byte. For example, if you have an array, you count each element separately. If you have a string, each element is only 1 byte, so no swapping is necessary. I've you have an array of shorts, ints, floats, etc. you will need to swap each individual item accordingly.
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #14
Jones Wrote:If I wrote this:

Code:
static bool CPU_BIG = FALSE;

#if defined(__BIG_ENDIAN__)
    CPU_BIG = TRUE;
#endif

And then checked what CPU_BIG was every time I read, it would be ok, si? Annoyed

Now you're doing a run-time test again, when you don't need to!

[spooky voice]Uuuse the #iiiiiiif...[/spooky voice]
Quote this message in a reply
Jones
Unregistered
 
Post: #15
OneSadCookie Wrote:Now you're doing a run-time test again, when you don't need to!

[spooky voice]Uuuse the #iiiiiiif...[/spooky voice]

Wh..wh...who... who said that? *terrified* Wink

You mean... declare the same function, but in one if it's with a conversion, and the other it isn't? Why? That just makes the code harder to read for n00bs like me with no understanding of such things. Rasp
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  Endian-ness ia3n_g 11 4,175 Sep 20, 2006 11:47 PM
Last Post: OneSadCookie
  Even *MORE* endian issues Jones 22 6,605 Aug 3, 2006 06:40 PM
Last Post: Jones