Hacking erroneous output of SBF Codec

[Tepee]

Last four mismatched bytes

In Post 52, Question 2, MotoCache1 asked about the last 4 unexplained mismached bytes. Using the good and bad monster.sbf files, SBF-Recalc shows two files for which data is calculated: RDL3 and CG42. It appears that SBF Codec calculates the 16-bit checksum (Check) for RDL3 correctly, but not for CG42. There is another field for these files, Wert, that differs between the good and bad files.

The values for Wert:

RDL3
Good 04419F75
_Bad 0441A11C

CG42
Good 003D7C87
_Bad 003D7DA6


The good numbers correspond to the 32-bit checksum values for the files. The upper four hex values are the same for both the good and bad, but the lower four mismatch. They're also the same as the last four unexplained mismatches as shown in Post #3.

It's interesting to note that the bad value for the 16-bit checksum for CG42 (7C87) is the same as the lower half of the correctly calculated 32-bit checksum (Wert value).

 

[Shibbey]

So, I guess that's the next question for you guys: How would you go about figuring out how to calculate the checksums for the Droid? The direct observation method I just described will work if you are very very good at math. Remember doing differential equations back in calculus and taking big formulas and having to factor them and blah blah blah -- this is kind of the reverse of that. All you are seeing are the answers and you have to figure out what formula is producing them.

Let me know what you come up with. It's possible I don't even have the answer and I just do the RSD Lite trick like everyone else does. Hard to say...

I found this on a German forum. As I am still new to the Droid hacking world, and am just starting to learn C#, maybe someone else can do something with this to help everyone out!

And here is the code for it

(Written in C #, but should serve well as pseudo code)

public static UInt16 GenerateSholsChecksum (byte [] bytes)
{ (
    UInt32 a = 0; UInt32 a = 0;
    UInt32 b = 0; UInt32 b = 0;

    var end = bytes.Length - (bytes.Length % 4); var end = bytes.Length - (bytes.Length% 4);

    for (var i = 0; i < end; i += 4) for (var i = 0; i <end; i + = 4)
    { (
        a += (UInt32)((bytes[i + 1] << 8) + bytes[i    ]); a + = (UInt32) ((bytes [i + 1] <<8) + bytes [i]);
        b += (UInt32)((bytes[i + 3] << 8) + bytes[i + 2]); b + = (UInt32) ((bytes [i + 3] <<8) + bytes [i + 2]);
    } )

    //if the bytes are not multiply of four, pad it with 0 / / If the bytes are not multiply of four, pad it with 0
    var diff = bytes.Length - end; var diff = bytes.Length - end;
    if (diff > 0) if (diff> 0)
    { (
        var nbytes = new byte[4]; var nbytes = new byte [4];

        for (var j = 0; j < 4; j++) for (var j = 0 j <4; j + +)
        { (
            if (j < diff) if (j <diff)
                nbytes[j] = bytes[end + j]; nbytes [j] = bytes [end + j];
            else else
                nbytes[j] = 0; nbytes [j] = 0;
        } )

        a += (UInt32)((nbytes[1] << 8) + nbytes[0]); = a + (UInt32) ((nbytes [a] <<8) + nbytes [0]);
        b += (UInt32)((nbytes[3] << 8) + nbytes[2]); b + = (UInt32) ((nbytes [3] <<8) + nbytes [2]);
    } )

    return (UInt16)(a + b + (a >> 16)); return (UInt16) (a + b + (a>> 16));            
} )

NOTE: For Yaffs2 partitions (they have code 0028) checksum 2048 bytes and you skip next 64 (spare parts).

EDIT: Actually, seeing the DROID sbf, the above applies for address greater than 0xD0000000 mainly.

 

[Tepee]

So, I guess that's the next question for you guys: How would you go about figuring out how to calculate the checksums for the Droid? The direct observation method I just described will work if you are very very good at math. Remember doing differential equations back in calculus and taking big formulas and having to factor them and blah blah blah -- this is kind of the reverse of that. All you are seeing are the answers and you have to figure out what formula is producing them.

Let me know what you come up with. It's possible I don't even have the answer and I just do the RSD Lite trick like everyone else does. Hard to say...

As I stated above, the values that SBF-ReCalc calls 'Wert' are simply the sum32 hash values. I don't know about the 16 bit 'Check' value. The 0x7C87 calculated by SBFCodec is sum16. I've tried to duplicate the 18BB value using almost a dozen 16-bit algorithms. I've tried:

adler16, crc16, crc16_ccit, crc16_ibm, crc16_x25, crc16_xmodem, crc16_zmodem, fletcher16, sum16, sumbsd and sumsysv.

No luck with any of them. Don't know about Shols. Have no idea what algorithm they're using to calculate the 18BB value.

 

[Shibbey]

Checksum

Has anyone figured it out yet? I am now 2.5 weeks late on schoolwork trying to learn all this and figure out the checksum! I hate the "oh shiny" effect I have on geek stuff. LOL

 

[Tepee]

What happened to this thread? it was supposed to be a tutorial on how to hack/debug/troubleshoot a problem creating sbf files.

Over in this topic we figured out that SBF Codec (the tool most folks use to try to make their own bootloader image SBF files) doesn't create the file correctly when it only has a single code group in it. This topic is going to be sort of a walk through of hacking that bad output to figure out how to make it good.

We were doing just fine until someone apparently solved the problem (Posts 32 & 34), but didn't share his findings. And that's where it ended. The rest of us kept plugging away but there haven't been any posts for almost a week! What gives? How about a clue?

 

[jlutz555]

What happened to this thread? it was supposed to be a tutorial on how to hack/debug/troubleshoot a problem creating sbf files.

Over in this topic we figured out that SBF Codec (the tool most folks use to try to make their own bootloader image SBF files) doesn't create the file correctly when it only has a single code group in it. This topic is going to be sort of a walk through of hacking that bad output to figure out how to make it good.

We were doing just fine until someone apparently solved the problem (Posts 32 & 34), but didn't share his findings. And that's where it ended. The rest of us kept plugging away but there haven't been any posts for almost a week! What gives? How about a clue?

Thanks for trying to throw me under the bus.....

But it was actually solved by posts 21 and 22. It took me till post 32 to figure that out.

Try re-reading the entire thread from the start.

 

[Jharmon12]

And I did share my results and I even posted a ss of what I done to make it work. The rest of this discussion was about the two questions about how to calc the checksum and the four mistery bytes.

Sent from my Droid using Tapatalk

 

[Tepee]

Sorry if it sounded like I was throwing you under the bus; I didn't intend to. Elegant prose is not my forte. I was simply trying to answer Shibbey's question about whether "anyone [has] figured it out yet".

I've read and re-read the posts in this thread. Post 21 makes no mention regarding determining the CG42 checksum other than to flash the bad .sbf file to obtain the correct value. Is that what you're doing? I had assumed (yeah, I know) that you had found a way to calculate it or read it somewhere in RSDLite/SBFCodec/SBFRecalc/FHRED. Post 22 explains Big and Little Endian, then suggests that there may be a way to calculate the checksum. This is also echoed in post 52. Checksums are explained then it's stated that the 'normal' method(s) aren't used - "they aren't calculated that way for the Droid." I know. I tried about a dozen 16-bit algorithyms with no luck. Other than that, I didn't see anything that gave me a flash of inspiration.

Post 68: "It would be nice if the phone used a simple checksum like the example at the beginning of this post. Unfortunately it does not. Exactly how the checksums are calculated is still an open question in this topic."


Post 74: "Sooo MC1 what tool do you use to make your sbf's?"


Post 79: "Has anyone figured it out yet? I am now 2.5 weeks late on schoolwork trying to learn all this and figure out the checksum!"

I was just wondering why this thread appeared to have died and maybe find out if anyone had progressed any further. No harm meant.

 

[Shibbey]

OK, so I ran into my first real complication using the proven method for determining checksum.

14:12:47,  September 27, 2010
Line: 1309
ERROR: Interface AP-OS: Error verifying Code Group 42 checksums. File: 0x73E3, Phone: 0xD72
File: X:\test_dev_usb\flash\code\flashdll\FlashOp.cpp
Device ID: 0

Am I missing something or is the checksum the phone wants missing a digit??

 

[jlutz555]

OK, so I ran into my first real complication using the proven method for determining checksum.

14:12:47,  September 27, 2010
Line: 1309
ERROR: Interface AP-OS: Error verifying Code Group 42 checksums. File: 0x73E3, Phone: 0xD72
File: X:\test_dev_usb\flash\code\flashdll\FlashOp.cpp
Device ID: 0

Am I missing something or is the checksum the phone wants missing a digit??

This was covered earlier...

Add a leading 0 to what the phone is expecting

0D 72

Then do little-big endian on it when you correct the sbf...

72 0D

and put it where you find

E3 73

HTH

 

[Shibbey]

Post 79: "Has anyone figured it out yet? I am now 2.5 weeks late on schoolwork trying to learn all this and figure out the checksum!"

I was just wondering why this thread appeared to have died and maybe find out if anyone had progressed any further. No harm meant.

I was refering to and am still curious about the 4 mystery bytes. I suspect the mystery was solved in This Post, but MotoCache has yet to confirm it.

I am also curious if anyone has figured out how to CALCULATE the checksum rather than failing the flash and fixing it.

 

[Shibbey]

This was covered earlier...

Add a leading 0 to what the phone is expecting

0D72

Then do little-big endian on it when you correct the sbf...

HTH

Thanks, I thought it was covered but scrolling back, I got a brain cramp trying to find it. LOL

 

[Shibbey]

My brain is cramping, but I am not going to give up yet! Currently researching that code I found on the German forum. Here is a link for those interested: Google Translate

 

[mhous33]

too bad i didn't find this thread until after i did the whole process manually lol; oh well, learned stuff along the way...
I don't have an answer for generating correct checksums, but just wanted to throw out there that when you use sbf_flash with linux, it tells you the incorrect and correct values before flashing the .sbf; might save a little time.
But if you're already using linux then you're probably as interested as i am to hear the rest of MotoCache1's comment here:

A super simple way to flash just CG42 on without even making an SBF also dawned on me, but it requires Linux, so I'll save that for a later time.

 

[Shibbey]

OK, so here is what I have done.....Way more than I can remember right now..but...

First: I loaded up the monster rebuild SBF in HxD. I then loaded the HMG file and both SMG file in HxD.

Discoveries:

• The SBF is a linear combination of the .HMG then RDL3 then CG42.

• HMG blocks 0-341
• ? blocks 342-352
• RDL3 blocks 353-4D352
• RDL3 sum32 hash - blocks 4D353-4D356
• ? blocks 4D357-4D36F
• CG42 blocks 4D370-8D36F
• CG42 sum32 hash - blocks 8D370-8D373
• ? blocks 8D374-8D37B

• There are 3 sections of mystery bytes that came from ??????. In the 2nd set of mystery bytes are two of the bytes we are still trying to figure out from comparing the good vs. the bad. Alas, the other two bytes are in my 3rd block of mystery bytes. Hmm... So, where do these come from?

My Mystery Blocks
Blocks 342-352:

Good: 02 00 00 00 00 00 03 02 01 00 04 D0 00 80 50 00 00
Bad: 02 00 00 00 00 00 03 02 01 00 04 D0 00 80 50 00 00
​

Blocks 4D353-4D36F: bold bytes are the RDL3 sum32 hash

Good: 04 41 9F 75 03 02 02 00 00 00 00 03 02 00 00 00 00 00 03 02 01 00 04 00 00 C0 58 00 00
Bad: 04 41 A1 1C 03 02 02 00 00 00 00 03 02 00 00 00 00 00 03 02 01 00 04 00 00 C0 58 00 00

​

Blocks 8D370-8D37B: bold bytes are the CG42 sum32 hash

Good: 00 3D 7C 87 03 02 02 00 00 00 00 03
Bad: 00 3D 7D A6 03 02 02 00 00 00 00 03
​

Next: I loaded up two SBF files that I made and did a comparison. The only differences I found were the date (known), the CG42 checksum (expected) and those two bytes in the second set of mystery blocks. But not in the third set! Why?


I think I have asked more questions than even came close to answering! I don't think I answered any questions but perhaps I am on the right path, or this may send someone on the right path for calculating all the rest automatically.

EDIT: I realized where some of the mystery bytes came from. Update info above! AND SOLVED THE RIDDLE OF THE LAST 4 DIFFERENT BYTES BETWEEN MONSTER GOOD AND BAD!