How to shift bits in a 2-5 byte long bytes object in python?

Don't use bytes to represent integer values; if you need bits, convert to an int:

value = int.from_bytes(your_bytes_value, byteorder='big')bits_21_to_8 = (value & 0x1fffff) >> 8

where the 0x1fffff mask could also be calculated with:

mask = 2 ** 21 - 1

Demo:

>>> your_bytes_value = b'\x93\x4c\x00'>>> value = int.from_bytes(your_bytes_value, byteorder='big')>>> (value & 0x1fffff) >> 84940

You can then move back to bytes with the int.to_bytes() method:

>>> ((value & 0x1fffff) >> 8).to_bytes(2, byteorder='big')b'\x13L'

As you have a bytes string and you want to strip the right-most eight bits (i.e. one byte), you can simply it from the bytes string:

>>> b'\x93\x4c\x00'[:-1]b'\x93L'

If you want to convert that then to an integer, you can use Python’s struct to unpack it. As you correctly said, you need a fixed size to use structs, so you can just pad the bytes string to add as many zeros as you need:

>>> data = b'\x93\x4c\x00'>>> data[:-1]b'\x93L'>>> data[:-1].rjust(4, b'\x00')b'\x00\x00\x93L'>>> struct.unpack('>L', data[:-1].rjust(4, b'\x00'))[0]37708

Of course, you can also convert it first, and then shift off the 8 bits from the resulting integer:

>>> struct.unpack('>Q', data.rjust(8, b'\x00'))[0] >> 837708

If you want to make sure that you don’t actually interpret more than those 13 bits (bits 8 to 21), you have to apply the bit mask 0x1FFF of course:

>>> 37708 & 0x1FFF4940

(If you need big-endianness instead, just use <L or <Q respectively.)

If you are really counting the bits from left to right (which would be unusual but okay), then you can use that padding technique too:

>>> struct.unpack('>Q', data.ljust(8, b'\x00'))[0] >> 431206656

Note that we’re adding the padding to the other side, and are shifting it by 43 bits (your 3 bits plus 5 bytes for the padded data we won’t need to look at)

Another approach that works for arbitrarily long byte sequences is to use the bitstring library which allows for bitwise operations on bitstrings e.g.

>>> import bitstring>>> bitstring.BitArray(bytes=b'\x93\x4c\x00') >> 3BitArray('0x126980')