Course Information


Yao Xie, Email:

Office: CIEMAS 3424

Office Hours: Tuesday, Thursday, 2:50 - 3:50pm. CIEMAS 3426.

Tentative Syllabus


Course Information


Miao Liu, Email:

Office Hours: Monday, Wednesday, 5:30 - 6:30pm. CIEMAS 3431.


Tuesday, Thursday, 1:25 - 2:40pm.


Hudson Hall 207.


This class provides an introduction to information theory. Information measures: entropy, mutual information, relative entropy and differential entropy. These topics are connected to practical problems in communications, compression, and inference, including lossless data compression, Huffman coding, asymptotic equipartition property, channel capacity, Gaussian channels, rate distortion theory, and Fisher information. The class is appropriate for beginning graduate students in electrical engineering, computer science, and mathematics who have a background in probability. Prerequisite: ECE255 or equivalent, or consent of instructor.


Lectures will follow the textbook. The rest of the material are recommended and will be helpful for some of the lectures.

  • Textbook: Thomas M. Cover and Joy A. Thomas, Elements of Information Theory, second edition, John Wiley, 2006. The author, Professor Tom Cover, just passed away on March 26, 2012. He worked till the last moment of his life. This course also pays tribute to him, the glorious figure in information theory.

  • R. G. Gallager, Information Theory and Reliable Communication, Wiley, 1968.

  • IEEE Transactions on Information Theory, Special Issue on 50 years of Information Theory, Vol. 44, No. 6, 1998.

  • Raymond W. Yeung, A first course in information theory, Springer, 2002.