Real Binary editor

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
There are millions of binary editors that allow user to change the bit value inside a binary bit string.

But is there one that allow user to delete/insert/replace bits in a long string of binary data?
 

thatoneguy

Joined Feb 19, 2009
6,359
I also use Ultra-Edit. They have a Unix version too now.

In *nix, hexedit and others will also work, some of those are ported to win32, though I haven't tested them to give a suggestion.
 

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
It seems all the suggestions so far are byte/hex editor, not the one I wanted.

In these editors, one can edit a certain bit in a 8-bit byte data to either 1 or 0 but you cannot remove/add bit(s).

To make it plain, if I have a 129bit binary string, can I use the above software to remove the 66th bit from this string and end up with a 128bit string?
 

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
Yes you can do it with the one I suggested, and the other one also I think.
Hi t06afre,

I have visited the link you have posted but it seems I have to install the whole package before I can try out the feature.

Can you do a little test to see how the editor turns out?

1. Open the editor and input five ASCII character 3DUUU (0x33 0x44 0x55 0x55 0x55)

2. Save it as a file and reopen it

3. Delete the lower nibble of the first 0X33 so now the hex string becomes (0x34 0x45 0x55 0x55 0x5?)

4. Save the file and reopen

5. If the file content now contains ASCII 4EUU (0x34 0x45 0x55 0x55) then the editor works as you have mentioned

Thanks
 

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
No problem. To see for myself and to verify your claim, I have downloaded and installed the software.

Unfortunately the result is still NO. I can't delete part of a byte using the Hackman editor. :(
 

someonesdad

Joined Jul 7, 2009
1,583
eblc1388, I think you've asked an interesting question. If I understand it, you're looking for a hex editor that will allow you remove a nybble from a stream of bytes and, thus, save a number of bits in a file divisible by 4 but not 8.

I suspect you're going to run into a problem finding what you want, as most things are byte-oriented. Thus, for example, on a typical PC, you can't save a file with 127 binary bits -- it will have to have an even number of bytes in order to be saved on disk.

One model for what you want is the xxd hex dump tool that comes with vim. It lets you create an ASCII hex dump of a file, edit the contents, then use xxd again to convert the data back to a binary file.

It wouldn't be too hard to write a program that will effectively get you what you want. One design pattern (among many) would be to use, like xxd, an ASCII-based syntax that lets you specify the bits desired. Here's one possibility:

  1. Bytes would be specified as two hex digits as is commonly done in hex editors, ASCII dumps, etc.
  2. A string of bits would be prefaced by "0b" and a binary bit pattern represented by "0" and "1" characters.
Here's a possible stream of digits:

a31b 0b11010 ff00ff

The tool would read this string in, then parse on whitespace. All strings that don't begin with "0b" are hex and must have an even number of hex digits. These are converted to their binary bit patterns. When something prefaced by "0b" is encountered, it's interpreted as a binary representation of some bits.

This tool would then translate the above representation into a canonical representation of a stream of hex digits followed by (an optional) binary string of 7 or fewer bits.

This would be relatively simple to write in a language like C. If you stored the data back in a file, you're going to have to encode things in some fashion. This would probably mean a header that tells the reading program how many bits are represented by the file. The remainder of the file would then be a sequence of bytes with the last byte perhaps containing less than 8 bits. There are lots of little programming details like endianness, interpreting the "0b" syntax, bookkeeping of bits, etc., but these are tactical things easily addressed.

If your need was purely nybble oriented, then it would be even easier to write a program that would interpret a stream of single digit hex characters. But you'd still need a file format that would encode how many bits are in the file.

To use these data stored in this fashion, you'll have to have a bit-oriented consumer and a program to feed it the required number of bits.
 

t06afre

Joined May 11, 2009
5,934
You cant adress bit on a harddisk. The byte is the smallest thing you may adress. So your 129 bit files does not make sense. Your file will always be rounded up to 17 bytes. Of course inside this 17 bytes all bits may be shifted around. But it will always be 17 byte long.
 

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
eblc1388, I think you've asked an interesting question. If I understand it, you're looking for a hex editor that will allow you remove a nybble from a stream of bytes and, thus, save a number of bits in a file divisible by 4 but not 8.
The example of nybble is just a special case. The original requirement is to insert/remove bit(s).

Let me give an application for example. One has captured a stream of binary data but unfortunately the correct start bit is somewhere in between the msb and lsb of a single byte. If one can pad the bits or remove the bits, the whole data stream aligned correctly.

I just wonder if there is any software that is capable of doing just that.

There is nothing to do with byte or anything, the data is a stream of binary data, which is stored as bytes in the computer harddisk or memory. The number of bits doesn't has to be exact multiple of 8 because there can be don't care bits either at the front or at the back of the data stream.

You cant adress bit on a harddisk. The byte is the smallest thing you may adress. So your 129 bit files does not make sense. Your file will always be rounded up to 17 bytes. Of course inside this 17 bytes all bits may be shifted around. But it will always be 17 byte long.
Of course one can access bit on a harddisk, within a particular byte on a particular sector. There is absolutely no difficulty in writing a software to do what is required but there isn't anything ready made as there is only the odd chance that someone might need to use such a piece of software.

I can simply convert the binary data stream into a long string of 0x30 and 0x31 so they appears as ascii "0" or "1" inside a normal text editor and I can edit/insert/delete whatever I would like. Afterwards I can convert the file back into binary.

I just wonder if there is any existing software that do all that without any conversion on my side.
 

THE_RB

Joined Feb 11, 2008
5,438
Don't any of the hex editors let you display the data and edit it when shown as binary? I seem to remember one of the older ones did, it might have been a DOS one.
 

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
Don't any of the hex editors let you display the data and edit it when shown as binary? I seem to remember one of the older ones did, it might have been a DOS one.
Yes, every hex editor does.

However, users can edit the data(i.e. change bit value) at bit level but are restricted to only insert or delete a whole byte(s). That's 8 bits at least.

There isn't a commonly known one that allow the user to insert or delete a single bit.
 

someonesdad

Joined Jul 7, 2009
1,583
OK, lemme rephrase my previous response -- as far as I know, a commercial tool that does what you want doesn't exist. That doesn't mean one doesn't exist, just that I haven't heard of it. The demand for such a tool would be very low. Besides, it's pretty easy to hack something together to do what you want for the odd special case when it comes up. If you need to do a lot of this, then it would make sense to put together a true editor (if it was me that needed it, I'd write the tool in wxPython, as most of the GUI tools are already there).
 

AlexR

Joined Jan 16, 2008
732
It all seems a bit pointless to me. Even if you could edit a binary word and add or delete bits what would you do with the edited product? You can't write it to a file without padding it out to a whole number of bytes. Computer storage operates on bytes rather than on bits and there is no way to store a data that is not composed of full bytes.
 

THE_RB

Joined Feb 11, 2008
5,438
You just pad the end of the file.

Your hard drive (ie OS) probably pads all files out to 32 byte or 64 byte boundary anyway, no file is ever in "bytes" anymore even though they pretend to be.
 

AlexR

Joined Jan 16, 2008
732
Well if you are going to pad the edit out to a whole byte why do you need an editor that can add or delete bits? Just use any hex editor and edit the bytes in the normal fashion.
 

Thread Starter

eblc1388

Joined Nov 28, 2008
1,542
The demand for such a tool would be very low. Besides, it's pretty easy to hack something together to do what you want for the odd special case when it comes up.
Yes, I agree with you that's the main reason why there isn't one known to many of us.

If someone needs to do that to a serial data stream, probably he/she would have sufficient knowledge to use a combination of other tools to get the job done. e.g. convert the data into 0x30 and 0x31 and then do the editing.
 
Top