Module CS4575-KP04

Duration

1 Semester

Turnus of offer

every summer semester

Credit points

4

Course of studies, specific fields and terms:

• Master Computer Science 2019, optional subject, Elective
• Master Medical Informatics 2019, optional subject, Medical Data Science / Artificial Intelligence
• Master Psychology 2016, optional subject, Elective
• Master Biophysics 2023, optional subject, Elective
• Master Media Informatics 2020, optional subject, Elective
• Master MES 2020, optional subject, Elective
• Master Entrepreneurship in Digital Technologies 2020, optional subject, specific
• Master Psychology - Cognitive Systems 2022, optional subject, psychology
• Master Psychology - Cognitive Systems 2027, optional subject, psychology

Classes and lectures:

• CS4575-V: Sequence Learning (lecture, 2 SWS)
• CS4575-Ü: Sequence Learning (exercise, 1 SWS)

Workload:

• 75 hours private studies
• 45 hours in-classroom work

Contents of teaching:

• Introduction to Sequence Learning (Formalisms, Metrics, Recapitulation of Relevant Machine Learning Techniques)
• Recurrent Neural Networks (Simple RNN Models, Backpropagation Through Time)
• Gated Recurrent Networks (Vanishing Gradient Problem in RNNs, Long Short-Term Memories, Gated Recurrent Units, Stacked RNNs)
• Important Techniques for RNNs (Teacher Forcing, Scheduled Sampling, h-Detach)
• Bidirectional RNNs and related concepts
• Hierarchical RNNs and Learning on Multiple Time Scales
• Online Learning and Learning without BPTT (Real-Time Recurrent Learning, e-Prop, Forward Propagation Through Time)
• Reservoir Computing (Echo State Networks, Deep ESNs)
• Spiking Neural Networks (Spiking Neuron Models, Learning in SNNs, Neuromorphic Computing, Recurrent SNNs)
• Temporal Convolution Networks (Causal Convolution, Temporal Dilation, TCN-ResNets)
• Introduction to Transformers (Sequence-to-Sequence Learning, Basics on Attention, Self-Attention and the Query-Key-Value Principle, Large Language Models)
• State Space Models (Structured State Space Sequence Models, Mamba)

Qualification-goals/Competencies:

• Students get a comprehensive understanding of most relevant sequence learning approaches
• Students learn to analyze the challenges in sequence learning tasks and to identify well-suited approaches to solve them
• Students will understand the pros and cons of various sequence learning models
• Students can implement common and custom sequence learning models for time series analysis, classification, and forecasting
• Students know how to analyze the models and results, to improve the model parameters, and to interpret the model predictions and their relevance

Grading through:

• Written or oral exam as announced by the examiner

Responsible for this module:

• Prof. Dr. Sebastian Otte

Teacher:

• Institute for Robotics and Cognitive Systems
• MitarbeiterInnen des Instituts
• Prof. Dr. Sebastian Otte

Literature:

• Goodfellow, I., Bengio, Y., & Courville, A. (2016) : Deep Learning MIT Press. ISBN 978-0262035613
• Prince, S. J. D. (2023) : Understanding Deep Learning The MIT Press. ISBN 978-0262048644
• Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020) : Mathematics for Machine Learning Cambridge University Press, 2020. ISBN 978-1108470049
• Nakajima, K., & Fischer, I. (2021) : Reservoir Computing: Theory, Physical Implementations, and Applications Cambridge University Press, 2020. ISBN 978-1108470049
• Sun, R., & Giles, C. (2001) : Sequence Learning: Paradigms, Algorithms, and Applications Springer Berlin Heidelberg. ISBN 978-3540415978
• Bishop, C. M. (2006) : Pattern Recognition and Machine Learning Springer. ISBN 978-0387310732
• Recent publications on the related topics :

Language:

• offered only in English

Notes:

Last Updated:

16.07.2025