Università degli Studi di Pavia

Facoltà di Ingegneria


Deep Learning

A.A. 2021-2022

Second Semester

Fri: 11:00 a.m. - 1:00 p.m., Aula E2

Fri: 2:00 p.m. - 4:00 p.m., Aula B2

Lectures & Suggested Readings:

  • Reports of errors in the resources below are always welcome
    1. 2022.03.11 (theory)

      Introduction [pdf]
      AI spring? Artificial Intelligence, Machine Learning, Deep Learning: facts, myths and a few reflections.

    2. 2022.03.11 (theory)

      Fundamentals: Artificial Neural Networks [pdf]
      Foundations of machine learning: dataset, representation, evaluation, optimization. Feed-forward neural networks as universal approximators.

    3. 2022.03.18 (theory)

      Flow Graphs and Automatic Differentiation [pdf]
      Tensorial representation, flow graphs. Automatic differentiation: graphs from graphs.

      Aside 1: Tensor broadcasting [pdf]
      From theoretical tensor algebra to actual computation: operators and automatic broadcasting.

    4. 2022.03.25 (theory)

      Deep Networks [pdf]
      Deeper networks: potential advantages and new challenges. Tensorial layerwise representation. Softmax and cross-entropy.

      Shannon Entropy (Wikipedia)

      Cross Entropy (Wikipedia)

    5. 2022.04.01 (theory)

      Learning as Optimization [pdf]
      Vanishing and exploding gradients. First and second order optimization, approximations, optimizers. Further tricks.

      Aside 2: Exponential Moving Average [pdf]

      Aside 3: Predictors [pdf]
      From in-sample optimization to out-of-sample generalization.

    6. 2022.04.22 (theory)

      Convolutional Networks [pdf]
      Convolutional filter, filter banks, feature maps, pooling, layerwise gradients.

      Aside 4: Hardware for Deep Learning [pdf]
      Main differences bewtween CPUs and GPUs, SIMT parallelism, bus-oriented communication, a few caveats.

    7. 2022.05.29 (theory)

      Deep Convolutional Neural Networks and Beyond [pdf]
      Some insight into what happens in convolution layers. DCNN architectures. Transfer learning. Working in reverse: image generation. Generative adversarial networks. Autoencoders and segmentation. Object detection.

    Instructor

    1. Marco Piastra

    2. Contact: marco.piastra@unipv.it


    Kiro

    1. Course info


    Exams

    1. See Faculty website


    Further resources:

    All Colab notebooks used in the course are available on Kiro


    Links

    1. Artificial Intelligence Reading Group


    1. Deep Learning, A.A. 2020-2021