hello y'all. as in the title iam confused about how mutual information intuitively relates to channel capacity. Take an example ,Let us have three random bits, X, Y, Z. X and Y are independent and unbiased, while Z is the exclusive or of X and Y. Then H(X,Y)=2, while H(X,Y|Z=z)=1 for any z∈{0,1} because knowing Z and the value of X determines Y. Then I(X,Y;Z)=H(X,Y)−H(X,Y|Z)=1. In words, if you know Z you only really need to encode X rather than both X and Y. So basically in send complete information about the system (X,Y) , if i know Z i should only encode H(X,Y|Z) and that's enough! So I save he trouble in encoding 1 bit or I(X;Y) bits to represent the entire (X,Y) system. Now how does Channel capacity come into the picture?
Last edited: