Reviewing the latest research in machine learning and intelligent systems. Your lecturers are Prof. Dr. Nils Bertschinger, Prof. Dr. Matthias Kaschube and Prof. Dr. Visvanathan Ramesh.

Time: Thursday, 8:00 am c.t.  SR 9

For any questions please contact us teaching@ccc.cs.uni-frankfurt.de

General

Bachelors students are required to give a presentation only, Masters also need to hand in a report about their topic (~ 5-10 pages), weighting for grade 50/50. Presentations will be around 45 minutes plus discussion with the class. Course language is English. We will meet every week and presence is mandatory. Either choose one of the topics below and search for literature for yourself (papers, book-chapters, etc.), or choose a paper from here or request one from any professor above. Registration is mandatory and will be passed to the examination-office.

Presentation Dates

• 27.04.2017 – Decisions on topics and dates
• 04.05.2017 – no seminar
• 11.05.2017 – no seminar
• 18.05.2017 –
• 25.05.2017 – Holiday (no seminar)
• 01.06.2017 –  no seminar
• 08.06.2017 –  Markus Ernst
• 15.06.2017 –  no seminar
• 22.06.2017 –  no seminar
• 29.06.2017 – no seminar
• 06.07.2017 – Leonidas Devetzidis
• 13.07.2017 – no seminar
• 20.07.2017 – Ludovica Sacco

Topics

• General introduction: Review papers of ML
• Deep Learning: Modern neural networks
• Classics:
  • Neural networks
  • Standard algorithms of ML, e.g. EM
• Applications
  • Neuroscience
  • Finance/Economics
  • Medicine
  • Computer vision (e.g. face recognition, action recognition)
  • Robotics (e.g. autonomous cars)
  • Recommendation systems (network models)
  • Natural language processing
• Systems/Architectures
  • Integration of subsystems
  • Platforms/Probabilistic programming (e.g. Tensorflow, Theano, Torch, EdwardLib, etc.)
  • Big Data
• Theoretical background:
  • Probability theory: Bayesian decision theory
  • Neural networks: Universal approximation

List of papers

* General philosophy
  Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models
  http://www.annualreviews.org/doi/pdf/10.1146/annurev-statistics-022513-115657
  Model-based machine learning
  https://www.microsoft.com/en-us/research/wp-content/uploads/2016/05/Bishop-MBML-2012.pdf

* Classics:
  - Neural networks
    A Sociological Study of the Official History of the Perceptrons Controversy
    http://www.jstor.org/stable/pdf/285702.pdf
  - Standard algorithms
    Maximum Likelihood from Incomplete Data via the EM Algorithm
    https://www.jstor.org/stable/pdf/2984875.pdf
    Variational Inference: A Review for Statisticians
    https://arxiv.org/pdf/1601.00670.pdf
    - Sampling algorithms
      MCMC Using Hamiltonian Dynamics
      A Conceptual Introduction to Hamiltonian Monte Carlo
      https://arxiv.org/abs/1701.02434
      Elliptical slice sampling
      http://www.jmlr.org/proceedings/papers/v9/murray10a/murray10a.pdf
      
* Applications
  - Finance/Economics
    - Volatility modeling:
      Bayesian analysis of GARCH and stochastic volatility: modeling leverage, jumps and heavy-tails for financial time series
      https://stat.duke.edu/research/BEST/BEST2009/NakajimaBEST2009honorablemention.pdf
      Generalized Wishart processes
      https://dslpitt.org/uai/papers/11/p736-wilson.pdf
    - Macroeconomics/Econometrics:
      Large Bayesian Vector Autoregressions
      http://onlinelibrary.wiley.com/doi/10.1002/jae.1137/pdf
    - Networks:
      Bayesian Models of Graphs, Arrays and Other Exchangeable Random Structures
      http://danroy.org/papers/OR-exchangeable.pdf
      Random function priors for exchangeable arrays with applications to graphs and relational data
      http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2012_0487.pdf
      Stochastic blockmodels and community structure in networks
      http://www-personal.umich.edu/~mejn/papers/blockmodel.pdf

* Systems/Architectures
  - Probabilistic programming
    - Stan (http://mc-stan.org)
      Stan: A probabilistic programming language for Bayesian inference and optimization
      http://www.stat.columbia.edu/~gelman/research/published/stan_jebs_2.pdf
    Black-Box Stochastic Variational Inference in Five Lines of Python
    http://people.seas.harvard.edu/~dduvenaud/papers/blackbox.pdf
    Deep Probabilistic Programming
    https://arxiv.org/abs/1701.03757

* Theoretical background
  - Deep learning
    Avoiding pathologies of deep architectures
    http://www.jmlr.org/proceedings/papers/v33/duvenaud14.pdf
    On Random Weights and Unsupervised Feature Learning
    http://www.robotics.stanford.edu/~ang/papers/nipsdlufl10-RandomWeights.pdf
    Dropout as a Bayesian Approximation: Insights and Applications
    http://mlg.eng.cam.ac.uk/yarin/PDFs/Dropout_as_a_Bayesian_approximation.pdf

* Representations and Computer Vision
    Equivariance, Invariance, Equivalence
    https://arxiv.org/pdf/1411.5908v2.pdf
    Learning Features Equivariant to Egomotion
    https://arxiv.org/pdf/1505.02206v1.pdf
    A Comparative Evaluation of Approximate Probabilistic Simulation and Deep Learning
    https://arxiv.org/pdf/1605.01138.pdf
    Residual Nets
    https://arxiv.org/pdf/1512.03385v1.pdf
    Learning representations via Curiosity https://arxiv.org/pdf/1604.01360v2.pdf

    LeCun Y, Bottou L, Bengio Y, Haffner P. 
    Gradient-based learning applied to document recognition. 
    Proceedings of the IEEE. 1998 Nov;86(11):2278-324.

   Chen X, Lawrence Zitnick C. Mind's eye: 
   A recurrent visual representation for image caption generation. 
   In Proceedings of the IEEE Conference on Computer Vision and 
     Pattern Recognition 2015 (pp. 2422-243)

 Donahue J, Anne Hendricks L, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, 
 Darrell T. Long-term recurrent convolutional networks for visual recognition and 
  description. InProceedings of the IEEE conference on computer vision and pattern 
 recognition 2015 (pp. 2625-2634).