Real Binary editor

Discussion in 'Programmer's Corner' started by eblc1388, Nov 23, 2009.

  1. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    There are millions of binary editors that allow user to change the bit value inside a binary bit string.

    But is there one that allow user to delete/insert/replace bits in a long string of binary data?
     
  2. beenthere

    Retired Moderator

    Apr 20, 2004
    15,815
    282
  3. thatoneguy

    AAC Fanatic!

    Feb 19, 2009
    6,357
    718
    I also use Ultra-Edit. They have a Unix version too now.

    In *nix, hexedit and others will also work, some of those are ported to win32, though I haven't tested them to give a suggestion.
     
  4. t06afre

    AAC Fanatic!

    May 11, 2009
    5,939
    1,222
  5. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    It seems all the suggestions so far are byte/hex editor, not the one I wanted.

    In these editors, one can edit a certain bit in a 8-bit byte data to either 1 or 0 but you cannot remove/add bit(s).

    To make it plain, if I have a 129bit binary string, can I use the above software to remove the 66th bit from this string and end up with a 128bit string?
     
  6. t06afre

    AAC Fanatic!

    May 11, 2009
    5,939
    1,222
    Yes you can do it with the one I suggested, and the other one also I think.
     
  7. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    Hi t06afre,

    I have visited the link you have posted but it seems I have to install the whole package before I can try out the feature.

    Can you do a little test to see how the editor turns out?

    1. Open the editor and input five ASCII character 3DUUU (0x33 0x44 0x55 0x55 0x55)

    2. Save it as a file and reopen it

    3. Delete the lower nibble of the first 0X33 so now the hex string becomes (0x34 0x45 0x55 0x55 0x5?)

    4. Save the file and reopen

    5. If the file content now contains ASCII 4EUU (0x34 0x45 0x55 0x55) then the editor works as you have mentioned

    Thanks
     
  8. t06afre

    AAC Fanatic!

    May 11, 2009
    5,939
    1,222
    I have not the time to do your test. But you can both insert,edit, and delete
     
  9. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    No problem. To see for myself and to verify your claim, I have downloaded and installed the software.

    Unfortunately the result is still NO. I can't delete part of a byte using the Hackman editor. :(
     
  10. someonesdad

    Senior Member

    Jul 7, 2009
    1,585
    141
    eblc1388, I think you've asked an interesting question. If I understand it, you're looking for a hex editor that will allow you remove a nybble from a stream of bytes and, thus, save a number of bits in a file divisible by 4 but not 8.

    I suspect you're going to run into a problem finding what you want, as most things are byte-oriented. Thus, for example, on a typical PC, you can't save a file with 127 binary bits -- it will have to have an even number of bytes in order to be saved on disk.

    One model for what you want is the xxd hex dump tool that comes with vim. It lets you create an ASCII hex dump of a file, edit the contents, then use xxd again to convert the data back to a binary file.

    It wouldn't be too hard to write a program that will effectively get you what you want. One design pattern (among many) would be to use, like xxd, an ASCII-based syntax that lets you specify the bits desired. Here's one possibility:

    1. Bytes would be specified as two hex digits as is commonly done in hex editors, ASCII dumps, etc.
    2. A string of bits would be prefaced by "0b" and a binary bit pattern represented by "0" and "1" characters.
    Here's a possible stream of digits:

    a31b 0b11010 ff00ff

    The tool would read this string in, then parse on whitespace. All strings that don't begin with "0b" are hex and must have an even number of hex digits. These are converted to their binary bit patterns. When something prefaced by "0b" is encountered, it's interpreted as a binary representation of some bits.

    This tool would then translate the above representation into a canonical representation of a stream of hex digits followed by (an optional) binary string of 7 or fewer bits.

    This would be relatively simple to write in a language like C. If you stored the data back in a file, you're going to have to encode things in some fashion. This would probably mean a header that tells the reading program how many bits are represented by the file. The remainder of the file would then be a sequence of bytes with the last byte perhaps containing less than 8 bits. There are lots of little programming details like endianness, interpreting the "0b" syntax, bookkeeping of bits, etc., but these are tactical things easily addressed.

    If your need was purely nybble oriented, then it would be even easier to write a program that would interpret a stream of single digit hex characters. But you'd still need a file format that would encode how many bits are in the file.

    To use these data stored in this fashion, you'll have to have a bit-oriented consumer and a program to feed it the required number of bits.
     
  11. t06afre

    AAC Fanatic!

    May 11, 2009
    5,939
    1,222
    You cant adress bit on a harddisk. The byte is the smallest thing you may adress. So your 129 bit files does not make sense. Your file will always be rounded up to 17 bytes. Of course inside this 17 bytes all bits may be shifted around. But it will always be 17 byte long.
     
  12. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    The example of nybble is just a special case. The original requirement is to insert/remove bit(s).

    Let me give an application for example. One has captured a stream of binary data but unfortunately the correct start bit is somewhere in between the msb and lsb of a single byte. If one can pad the bits or remove the bits, the whole data stream aligned correctly.

    I just wonder if there is any software that is capable of doing just that.

    There is nothing to do with byte or anything, the data is a stream of binary data, which is stored as bytes in the computer harddisk or memory. The number of bits doesn't has to be exact multiple of 8 because there can be don't care bits either at the front or at the back of the data stream.

    Of course one can access bit on a harddisk, within a particular byte on a particular sector. There is absolutely no difficulty in writing a software to do what is required but there isn't anything ready made as there is only the odd chance that someone might need to use such a piece of software.

    I can simply convert the binary data stream into a long string of 0x30 and 0x31 so they appears as ascii "0" or "1" inside a normal text editor and I can edit/insert/delete whatever I would like. Afterwards I can convert the file back into binary.

    I just wonder if there is any existing software that do all that without any conversion on my side.
     
  13. THE_RB

    AAC Fanatic!

    Feb 11, 2008
    5,435
    1,305
    Don't any of the hex editors let you display the data and edit it when shown as binary? I seem to remember one of the older ones did, it might have been a DOS one.
     
  14. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    Yes, every hex editor does.

    However, users can edit the data(i.e. change bit value) at bit level but are restricted to only insert or delete a whole byte(s). That's 8 bits at least.

    There isn't a commonly known one that allow the user to insert or delete a single bit.
     
  15. someonesdad

    Senior Member

    Jul 7, 2009
    1,585
    141
    OK, lemme rephrase my previous response -- as far as I know, a commercial tool that does what you want doesn't exist. That doesn't mean one doesn't exist, just that I haven't heard of it. The demand for such a tool would be very low. Besides, it's pretty easy to hack something together to do what you want for the odd special case when it comes up. If you need to do a lot of this, then it would make sense to put together a true editor (if it was me that needed it, I'd write the tool in wxPython, as most of the GUI tools are already there).
     
  16. AlexR

    Well-Known Member

    Jan 16, 2008
    735
    54
    It all seems a bit pointless to me. Even if you could edit a binary word and add or delete bits what would you do with the edited product? You can't write it to a file without padding it out to a whole number of bytes. Computer storage operates on bytes rather than on bits and there is no way to store a data that is not composed of full bytes.
     
  17. THE_RB

    AAC Fanatic!

    Feb 11, 2008
    5,435
    1,305
    You just pad the end of the file.

    Your hard drive (ie OS) probably pads all files out to 32 byte or 64 byte boundary anyway, no file is ever in "bytes" anymore even though they pretend to be.
     
  18. AlexR

    Well-Known Member

    Jan 16, 2008
    735
    54
    Well if you are going to pad the edit out to a whole byte why do you need an editor that can add or delete bits? Just use any hex editor and edit the bytes in the normal fashion.
     
  19. eblc1388

    Thread Starter Senior Member

    Nov 28, 2008
    1,542
    102
    Yes, I agree with you that's the main reason why there isn't one known to many of us.

    If someone needs to do that to a serial data stream, probably he/she would have sufficient knowledge to use a combination of other tools to get the job done. e.g. convert the data into 0x30 and 0x31 and then do the editing.
     
  20. BMorse

    Senior Member

    Sep 26, 2009
    2,675
    234

    For PC or Microcontroller??
     
Loading...