ECE587: Information Theory

This is the website for ECE 587, Duke University, Fall semester 2012.

These lecture notes is a tribute to the beloved Thomas M. Cover, from whom I learned information theory.

Information theory is the science of operations on data. The central themes of information theory include compression, storage, and communication. The birth of information theory was in 1948, marked by Claude E. Shannon's paper entitled "A Mathematical Theory of Communication". Surprisingly, this paper paved way for many later revolutionary ideas such as digital communication, storage and even internet. Our course will explore the basic concepts of Information theory. It is highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. Also encouraged to enroll are students who are interested in seeing the interplay between information theory and probability theory.

This class provides an introduction to information theory. Information measures: entropy, mutual information, relative entropy and differential entropy. These topics are connected to practical problems in communications, compression, and inference, including lossless data compression, Huffman coding, asymptotic equipartition property, channel capacity, Gaussian channels, rate distortion theory, and Fisher information. The class is appropriate for beginning graduate students in electrical engineering, computer science, and mathematics who have a background in probability. Prerequisite: ECE255 or equivalent, or consent of instructor.

Announcements