EE 568

Elements of Information Theory

Instructor: Natalia A. Schmid 

Spring 2007

Course Syllabus

Announcements:  

Lecture Notes: 

·         Lecture Notes from January 11, 2007 [handouts]

Recommended Books and Papers: 

        Required: 

·         T. M. Cover and J.A. Thomas, Elements of Information Theory, John Wiley & Sons, New York, 1991.

            Collateral Textbook: 

·         R. G. Gallager,  Information Theory and Reliable Communication, John Wiley & Sons, 1968.

·         R. E. Blahut, Principles and Practice of Information Theory, Addison-Wesley, 1988. 

·         S. Kullback, Information Theory and Statistics, Dover, 1997.

·         J. A. Bucklew, Large Deviation Techniques in Decision, Simulation, and Estimation, New York: Wiley, 1990.

 

            Kolmogorov Complexity: 

 

·         A. N. Kolmogorov, “Logical Basis for Information Theory and Probability Theory,” IEEE Trans. on Info. Theory, vol. IT-14, no. 5, Sept. 1968 [pdf-copy].

·         G. J. Chaitin, “On the Difficulty of Computations,” IEEE Trans. on Info. Theory, vol. IT-16, no.1, Jan. 1970 [pdf-copy].

·         G. J. Chaitin, “Informatio-Theoretical Computational Complexity,” IEEE Trans. on Info. Theory, vol. IT-20, no.1, Jan. 1974 [pdf-copy].

 

            MDL: 

 

·         Grunwald’s tutorial on MDL [pdf-copy].  

·         A. Lanterman’s Review on MDL and MML [pdf-copy].

·         Barron, A.; Rissanen, J.; Bin Yu; “The minimum description length principle in coding and modeling,” IEEE Trans. on Info. Theory, vol. 44 , no. 6 , Oct. 1998, pp. 2743 – 2760 [pdf-copy].
 

            Rate Distortion: 

 

·         T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression, Prentice Hall, Englewood Cliffs, NJ, 1971.

·         A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Publishers, Boston, 1992.

·         See the link on Vector Quantization.

·         Robert Gray’s web page http://www-ee.stanford.edu/~gray/

 

Useful Links: 

·         IEEE Information Theory Society: http://www.itsoc.org/

·         Elements of Information Theory: http://www-isl.stanford.edu/~jat/eit2/newpage3.htm

·         G. J. Chaitin, personal web page: http://www.cs.auckland.ac.nz/CDMTCS/chaitin/ 

·         MDL research web page: http://www.mdl-research.org/

·         Vector Quantization: http://www.data-compression.com/vq.shtml

 

 



This file is maintained by Natalia A. Schmid.

Respond to her via natalias@csee.wvu.edu