What kind of "checksum" are you are passing from PC to PIC and back as the output from your PRNG? A simple checksum checks for content integrity not it's uniqueness and CRC type functions are not designed to be hashes or used for key derivation (a public reseeding token/key in your case whose predictability could compromise the dongle).
http://home.comcast.net/~bretm/hash/8.html


I assume with your PIC dongle it's possible to have two different inputs with the same pre-reseeded generator bits, which have the same output token from 32bit collisions in the compression of much greater than 32 random bits (I assume it's from cache data result after being mangled). A proper cryptographic hash function handles collisions in a secure way and is designed to be collision-resistant.

 

How is the PIC initialized after sequencing checksums from one file with it's key to the next file with a different key? How does it know to erase past history so the send/receive PICs can synchronize the new keystream with new plaintext as is needed with a "synchronous stream cipher"?

What I had imagined was the PIC dongle would be plugged in, then on the first 32bit checksum it receives it would do the "initiialisation" process. Of course this assumes you plug in the PIC and then do a file encrypt or decrypt.

If you think that is clunky, then it would be easy to detect if say a single 0x55 byte is received (as a "command") instead of a 32bit checksum, and if it is the command then the PIC can re-initialise.

The PIC initialising erases all its internal "past history".

What kind of "checksum" are you are passing from PC to PIC and back as the output from your PRNG? A simple checksum checks for content integrity not it's uniqueness and CRC type functions are not designed to be hashes or used for key derivation (a public reseeding token/key in your case whose predictability could compromise the dongle).
http://home.comcast.net/~bretm/hash/8.html
...

Good point, and I welcome all suggestions here. The goal of the checksum is just to represent a unique 32bit value based on the contents of the 1Mbit cache or the 8kbit cache. The checksum could also be an output of 32 separate bits extracted as the RNG runs.

I had not bothered thinking too much about checksums as I imagined it would be trivial to make 32bits from 1Mbit or 8kbit, without being able to reverse the 32bits back to the cache!

Thanks for the link on CRC hash issues, I just read that. I was not planning to use a CRC system, but was probably going to extract 32bits in sequence fairly evenly spaced from the cache. Although this does not represent the entire content of the cache the bits will be unique each time, as (after scrambling)every bit in the cache is random and has an equal chance of 0 or 1.

Alternatively it could do a simple additive (or XOR) 8bit checksum made from all bytes in each quarter of the cache, that would represent all cache bits and give a unique 32bit checksum in total.

Personally I kind of like the idea of only using some of the cache to generate the 32bit checksum, this adds a layer of redundancy making it even harder to ghuess the cache contents from knowing a checksum.

Ultimately I think the goal is to make sure that a checksum could not be predicted by knowing the previous checksum. That should be easy to satisfy. Like I said I welcome suggestions for that.

I assume with your PIC dongle it's possible to have two different inputs with the same pre-reseeded generator bits, which have the same output token from 32bit collisions in the compression of much greater than 32 random bits (I assume it's from cache data result after being mangled). A proper cryptographic hash function handles collisions in a secure way and is designed to be collision-resistant.

Sorry but you completely lost me there. To put it in my words;
The PIC cache has 8kit random data, then the 32bit checksum is partially seeded in on top of that data.
Then the scrambling is done on the PIC cache, making a new 8kbit random cache, then that is "compressed" in some way to become the output 32bit checksum. The cache data remains, ready for next loop.

So yes, there is a chance two different input checksums would produce the same output checksum, but that chance is eactly one in 2^32, assuming of course the cache was sufficiently "randomised". I hope I understood your question correctly.

I'm not sure I understand what you mean by "collisions" but it appears you are talking about the different checksums in causing the same checksum out? I can't see how that compromises security beacuse the entire process uses humdreds of checksums, and any checksum has an equal chance of being any value.

To put it in other words, the entire job of the PIC dongle is to accept a checksum in, and produce a checksum out that can not be predicted (has an equal chance of being ANY value). In that case collisions will occur by chance in order with the odds of being random.

Remember too that becasue the cache data remains each cycle, it doesn't matter that input checksums are identical because the PIC will still produce different output checksums (ie "rolling code" type behaviour).

 

I think you do need to address the initialization because, almost certainly, you are going to have times that you want to encrypt several files and, sooner or later, you are going to forget to unplug the dongle and plug it back in. You could, presumable, have the PC software remind you to do it. You could also have a reset button on the dongle to make it easier to do (but it would still require manual intervention). Since your communication bandwidth is going to be so low because of the computational overhead, you could simply make a protocol that always sends 5 bytes (or even 8 bytes) and have the non-data bytes be a command byte. An alternative, though one with a theoretical risk, is to use a flagged checksum sequence. For instance, if you send it two All-Zero checksums in a row or if you send it an All-Zero followed by an All-One. The odds of two specific checksum sequences happening back to back is only 1 in 2^64, but there definitely is the possibility that it COULD happen. To detect if it did happen, you simply have the PC code monitor for it as well and, if it happens, you prompt the user that they must choose a different password because the one they chose happens to be poorly-behaved (and I believe all crypto systems have certain keys that are poorly behaved, in some cases we know what they are, but usually we only know that they do exist).

 

What I had imagined was the PIC dongle would be plugged in, then on the first 32bit checksum it receives it would do the "initiialisation" process. Of course this assumes you plug in the PIC and then do a file encrypt or decrypt.

If you want your system to be seamless then a remote "init" command is a must.

To put it in other words, the entire job of the PIC dongle is to accept a checksum in, and produce a checksum out that can not be predicted (has an equal chance of being ANY value). In that case collisions will occur by chance in order with the odds of being random.

My comments were a caution less directed specificity at your system but more at hashes in general to not to assume the "checksum" hashing function is actually random in the design of the randomness extractor. It's much better to have the collision probability dependent on the input rather than it being a main property of the hashing function to make it resistant to something like a "birthday attack".

 

I think, given his approach, he is going to have to generate a pseudo random bit stream that is a function of the password only, and not also of the input, if he is going to be able to decrypt because he doesn't have a Fiestal structure or anything else that would let him produce the same bit stream using the password and the ciphertext.

If he wanted to do that, probably the simplest way would be to use an autokey scheme.

 

I think you do need to address the initialization because, almost certainly, you are going to have times that you want to encrypt several files and, sooner or later, you are going to forget to unplug the dongle and plug it back in. You could, presumable, have the PC software remind you to do it.
...

Ok, remote initialisation is a must. I think the easy way is for the PC to send a single command byte (like 0x55) followed by a pause. The PIC will generate an error on any reception which is NOT a 4 byte checksum.

So if it receives a single byte, the PICs error handler would check if that byte is the 0x55 re-initialise command, and if so do an initialisation, and then wait for checksums to start again.

...My comments were a caution less directed specificity at your system but more at hashes in general to not to assume the "checksum" hashing function is actually random in the design of the randomness extractor. It's much better to have the collision probability dependent on the input rather than it being a main property of the hashing function to make it resistant to something like a "birthday attack".
...

Cool, I think we are on the same wavelength there. WOuld you like to provide some input on my point whether to generate the 32bit checksum from ALL of the cache data, or just derive the 32bit checksum from a small sampling of the cache?

Or keeping in the spirit of the Black RNG maybe generate the checksum 32bits in real time as the RNG operates, which would "randomly" selects 32bits from an unknown area or areas of the cache?

I think, given his approach, he is going to have to generate a pseudo random bit stream that is a function of the password only, and not also of the input, if he is going to be able to decrypt because he doesn't have a Fiestal structure or anything else that would let him produce the same bit stream using the password and the ciphertext.

If he wanted to do that, probably the simplest way would be to use an autokey scheme.

That's correct, the data itself doesn't affect the RNG process. Basically the two RNGs produce a psuedo random sequence based only on the password, not on the data. We addressed this earlier when NSAspook identified it as a potential problem, and I suggested the PC adds a random "autokey" of 128bits which is specific to the file being encrypted, and probably stored in that files header.

My post #19 describes what the dual RNGs use to generate the final data key.

 

Hi, I've started a simple hardware RNG to generate high quality natural random data for use in making seeds and keys for this project.

The hardware RNG can be seen on the forum here;
http://forum.allaboutcircuits.com/showthread.php?t=73520

It's almost finished and I'm currently generating data and testing variation on the output "whitening" algorithm to remove the small bias of 1 and 0 bits caused by natural processes.

Any suggestions for that would be appreciated.