Mention That nBits Can Be Negative

[harding]

Hi Krzysztof,

The description of nBits should probaly mention that the high bit of the
mantissa indicates sign---otherwise implementors might accidentally
encode a different target value than they intended. For example, an
nBits of 0x01803456 would be parsed by the formula in the reference as:

## Python interactive interpreter
hex(int(0x803456 * 256**(0x01-3))) == '0x80'

But the Bitcoin Core src/test/bignum_tests.cpp file says:

num.SetCompact(0x01803456);
BOOST_CHECK_EQUAL(num.GetHex(), "0");
BOOST_CHECK_EQUAL(num.GetCompact(), 0U);

There can even be negative nBits:

num.SetCompact(0x04923456);
BOOST_CHECK_EQUAL(num.GetHex(), "-12345600");
BOOST_CHECK_EQUAL(num.GetCompact(), 0x04923456U);

I'm pasting a draft copy of some text I've written about this for the
Bitcoin.org developer docs; it provides some more background and
explains how Bitcoin Core deals with the situation based on my reading
of the code (which could be wrong, of course). It also has references to
the relevant code. Please feel free to use any text you find useful; no
credit is required.

Before I start pasting, I want to thank you for producing the excellent
reference doc. It's always one of the first places I check when I'm
trying to figure something out.

Target nBits

The target threshold is a 256-bit unsigned integer compared the 256-bit
SHA256(SHA256()) header hash (treated also as an unsigned integer).
However, the header field nBits provides only 32 bits of space, so the
target number uses a less precise format called "compact". A naïve (but
incomplete) breakdown of nBits shows it to be a sort of scientific
notation to provide an approximate value.

TK: illustration
         0x181bc330

0x1bc330     *    256**(0x18-3)
significand      base  exponent
(mantissa)

Result: 0x1bc330000000000000000000000000000000000000000000

Padded to 32 bytes: 0x00000000000000001bc330000000000000000000000000000000000000000000

As a base-256 number, nBits can be quickly parsed as bytes the same way
you might parse a decimal number in base-10 scientific notation:

TK: illustration
0x181bc330

Byte length: 0x18 (decimal 24)

bytes  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17 18 19 20 21 21 23 24
     0x__ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __ __
MSB    1b c3 30

Most Significant Bytes (MSB): 0x1bc330

<!-- Source for paragraph below: Bitcoin Core src/tests/bignum_tests.cpp:
num.SetCompact(0x04923456);
BOOST_CHECK_EQUAL(num.GetHex(), "-12345600");
BOOST_CHECK_EQUAL(num.GetCompact(), 0x04923456U);
-->

Although the target threshold should be an unsigned integer, the nBits
header field uses a signed data type, allowing the target threshold to
be negative if the high bit of the significand is set. However, because
the header hash is treated as an unsigned number, it can never be equal
to or lower than a negative target threshold. Bitcoin Core deals with
this in two ways:

<!-- source for "Bitcoin Core converts..." src/main.h GetBlockWork() -->

• When parsing nBits, Bitcoin Core converts a negative target
  threshold into a target of zero, which the header hash can equal (in
  theory, at least).
• When creating a value for nBits, Bitcoin Core checks to see if it will
  produce an nBits which will be interpreted as negative; if so, it
  divides the significand by 256 and increases the exponent by 1 to
  produce the same number with a different encoding.

Some more examples taken from the Bitcoin Core test cases:

TK: table
Bits          Target
0x01003456 =  0x00
0x01123456 =  0x12
0x02008000 =  0x80
0x05009234 =  0x92340000
0x04923456 = -0x12345600   High bit set (0x80 in 0x92).
0x04123456 =  0x12345600   Inverse of above; no high bit.

[minium]

Hey David,

this is very strange. According to the block header definition, the nBits field is of an unsigned integer type, in 0.9.0 it was "unsigned int" and in the current code it is "uint32_t", to be precise. I'll look into that a bit deeper and see how it unfolds.

Warm greetings,
Krzysztof

[minium]

You are indeed correct. The compact floating point representation includes a sign bit.
The full target is computed from its compact representation as:

significand = compact & 0x007FFFFF
exponent = compact & 0xFF000000

this can be found exactly in here here.

The full target is then:

• significand * 2^(8*(3-exponent)) IF exponent <= 3
• significand * 2^(8*(exponent-3)) IF exponent > 3

When the next target is computed and stored in nBits, the sign bit is essentially ignored.
This can be seen in here. The new target is computed and when it is returned, the sign is always set to positive.

I'll adopt these changes asap. Thanks for the help.

Warm greetings,
Krzysztof

[harding]

@minium that description looks accurate to me. Thanks!