===== Information Theory & Coding ===== \\ |Instructor|**Emre Telatar**| |Office|[[http://plan.epfl.ch/?room=INR117|INR 117]]| |Phone|**+41 21 69 37693**| |Email|**emre.telatar@epfl.ch**| |Office Hours|**By appointment**| | | | | | | |Teaching Assistant |**Rajai Nasser**| |Phone|**+41 21 69 37554**| |Office|[[http://plan.epfl.ch/?room=INR036|INR 036]]| |Email|**rajai.nasser@epfl.ch**| |Office Hours|| | | | | | | |Teaching Assistant |**Serj Haddad**| |Phone|**+41 21 69 31357**| |Office|[[http://plan.epfl.ch/?room=INR038|INR038]]| |Email|**serj.haddad@epfl.ch**| |Office Hours|| \\ \\ Schedule to be confirmed |Lectures|**Monday** |**13:00 - 15:00** (Room: [[http://plan.epfl.ch/?reset_session&alias=ela1|ELA01]])| | |**Tuesday** |**13:00 - 15:00** (Room: [[http://plan.epfl.ch/?reset_session&alias=ela2|ELA02]])| |Exercises|**Tuesday** |**15:00 - 17:00** (Room: [[http://plan.epfl.ch/?reset_session&room=ela2|ELA02]])| \\ |**Language**:| |English| |**Credits **:| |7 ECTS| \\ See the course {{:en:courses:2013-2014:itc:info.pdf|information}}. /* [[http://isa.epfl.ch/imoniteur_ISAP/!itffichecours.htm?ww_i_matiere=1774097&ww_x_anneeAcad=2011-2012&ww_i_section=2139068&ww_i_niveau=&ww_c_langue=en|Link]] to official course description. */ ==== Special Announcements ==== Midterm results: {{:en:courses:2013-2014:itc:midtermresults.pdf|}}.\\ **FINAL EXAM ANNOUNCEMENT\\ The final exam has been set on Thursday January 16th from 16:15 to 19:15. It will take place in room PO 01. The exam will cover all the topics in the course. You can have twice the number of sheets as in the midterm. No book, no notes, no cellphones, no pocket calculators, or any other electronic devices.\\ Notes on Tunstall coding, the Lempel-Ziv algorithm, coding theory and polar coding are posted on “Additional Reading Material” section. ** ==== Detailed Schedule ==== \\ ^ Date ^ Topics Covered ^ Assignment ^ Solutions ^ Remarks ^ | Sep 16 | | | | public holiday, | | | | | | no course :-) | | | | | | | | | | | | | | | | | | | | Sep 17 | Intro to source coding; non-singular, | {{:en:courses:2013-2014:itc:hw01.pdf|Homework 1}} | {{:en:courses:2013-2014:itc:sol01.pdf|Solutions 1}} | | | | uniquely decodable, prefix-free codes; | | | | | | Kraft's inequality for prefix-free codes. | | | | | | | | | | | | | | | | | | | | | | | Sep 23 | Kraft's inequality for uniquely decodable | | | | | | codes; lower bound on optimal codes. | | | | | | | | | | | | | | | | | | | | | | | Sep 24 | Upper bound on optimal codes; |{{:en:courses:2013-2014:itc:hw02.pdf|Homework 2}}| {{:en:courses:2013-2014:itc:sol02.pdf|Solutions 2}}| | | | Huffman codes. | | | | | | | | | | | | | | | | | | | | | | | Sep 30 | Entropy (H); Properties of H; Joint entropy; | | | | | | Divergence; Conditional entropy; | | | | | | Mutual information (I); Chain rule for H. | | | | | | | | | | | | | | | | | | | | | | | Oct 01 | Conditioning reduces entropy; Properties |{{:en:courses:2013-2014:itc:hw03.pdf|Homework 3}}| {{:en:courses:2013-2014:itc:sol03.pdf|Solutions 3}}| | | | of I; Conditional mutual information; | | | | | | Chain tule for I; Interpretation of H.| | | | | | | | | | | | | | | | | | | | | | | Oct 07 | | | | | | | | | | | | | | | | | | | | | | | | Oct 08 | |{{:en:courses:2013-2014:itc:hw04.pdf|Homework 4}}| {{:en:courses:2013-2014:itc:sol041.pdf|Solutions 4}}| | | | | | | | | | | | | | | | | | | | | Oct 14 | | | | | | | | | | | | | | | | | | | | | | | | Oct 15 | |{{:en:courses:2013-2014:itc:hw051.pdf|Homework 5}}| {{:en:courses:2013-2014:itc:sol051.pdf|Solutions 5}}| | | | | | | | | | | | | | | | | | | | | Oct 21 | | | | | | | | | | | | | | | | | | | | | | | | Oct 22 | |{{:en:courses:2013-2014:itc:hw06.pdf|Homework 6}}| {{:en:courses:2013-2014:itc:sol06.pdf|Solutions 6}}| | | | | | | | | | | | | | | | | | | | | Oct 28 | | | | | | | | | | | | | | | | | | | | | | | | Oct 29 | |{{:en:courses:2013-2014:itc:midterm.pdf|Midterm}}| {{:en:courses:2013-2014:itc:midsol.pdf|Solutions}}| | | | | | | | | | | | | | | | | | | | | Nov 04 | | | | | | | | | | | | | | | | | | | | | | | | Nov 05 | |{{:en:courses:2013-2014:itc:hw07.pdf|Homework 7}}| {{:en:courses:2013-2014:itc:sol07.pdf|Solutions 7}}| | | | | | | | | | | | | | | | | | | | | Nov 11 | | | | | | | | | | | | | | | | | | | | | | | | Nov 12 | |{{:en:courses:2013-2014:itc:hw08.pdf|Homework 8}}| {{:en:courses:2013-2014:itc:sol08.pdf|Solutions 8}}| | | | | | | | | | | | | | | | | | | | | Nov 18 | | | | | | | | | | | | | | | | | | | | | | | | Nov 19 | |{{:en:courses:2013-2014:itc:hw09.pdf|Homework 9}}| {{:en:courses:2013-2014:itc:sol09.pdf|Solutions 9}}| | | | | | | | | | | | | | | | | | | | | Nov 25 | | | | | | | | | | | | | | | | | | | | | | | | Nov 26 | |{{:en:courses:2013-2014:itc:hw10.pdf|Homework 10}}| {{:en:courses:2013-2014:itc:sol10.pdf|Solutions 10}}| | | | | | | | | | | | | | | | | | | | | Dec 02 | | | | | | | | | | | | | | | | | | | | | | | | Dec 03 | | | | | | | | | | | | | | | | | | | | | | | | Dec 09 | | | | | | | | | | | | | | | | | | | | | | | | Dec 10 | |{{:en:courses:2013-2014:itc:hw11.pdf|Homework 11}}| {{:en:courses:2013-2014:itc:sol11.pdf|Solutions 11}}| | | | | | | | | | | | | | | | | | | | | Dec 16 | | | | | | | | | | | | | | | | | | | | | | | | Dec 17 | |{{:en:courses:2013-2014:itc:hw12.pdf|Homework 12}}| {{:en:courses:2013-2014:itc:sol12.pdf|Solutions 12}}| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Final exam 2012 |{{:en:courses:2013-2014:itc:final2012.pdf|Final}}| {{:en:courses:2013-2014:itc:finalsol12.pdf|Solutions}}| | | | | | | | | | | | | | | | | | | | | | Final exam 2013 |{{:en:courses:2013-2014:itc:final2013.pdf|Final}}| {{:en:courses:2013-2014:itc:finalsol13.pdf|Solutions}}| | | | | | | | | | | | | | | | | | | | | | Final exam 2014 |{{:en:courses:2013-2014:itc:final.pdf|Final}}| {{:en:courses:2013-2014:itc:final-sol.pdf|Solutions}}| | ==== Textbook ==== [[http://opac.nebis.ch/F?local_base=nebis&func=find-b&find_code=020&request=0-471-24195-4&con_lng=ENG|Elements of information theory]], Thomas M. Cover, Joy A. Thomas, 2006. ISBN:0-471-24195-4 ==== Additional Reading Material ==== Claude Shannon's {{:en:courses:2013-2014:shannon1948.pdf|A Mathematical Theory of Communication}}, published in Bell System Technical Journal, 1948 (part 1 in July, part 2 in October)\\ \\ **Notes on Tunstall codes**: {{:en:courses:2013-2014:itc:tunstall.pdf|Tunstall}}\\ **Notes on the Lempel-Ziv algorithm**: {{:en:courses:2013-2014:itc:lz.pdf|Lempel-Ziv}}\\ **Notes on coding theory**: {{:en:courses:2013-2014:itc:coding.pdf|Coding}}\\ **Presentation on polar codes**: {{:en:courses:2013-2014:itc:polar-notes.pdf|Polar}}\\