Seminar-DeepLearning-2018

Session 1 : Basic Neural Networks

17th May (Thrushday) 14:00 - 17:00 ; Room - 018 (Bâtiment Pascal)

Tanmoy Mondal : Post-Doctoral Researcher

Short-bio

Tanmoy Mondal received B.Tech. degree in Information Technology from West Bengal University of Technology, Kolkata, India, in 2007 and the M.Tech. degree in Mechatronics & Robotics from Bengal Engineering and Science University, Kolkata, in 2009. Before joining as a PhD student at Université de Tours (France) in 2012, he worked at several industries and premier R&D centers as a researcher. After completing his PhD from Laboratoire d’Informatique, Université François Rabelais, Tours, France in 2015, currently he did Post-Doc at INSA, Lyon, France. Currently, he is working as Post-Doc at L3i, University of La Rochelle. His research interests include pattern recognition, image processing and analysis, and computer vision. His current research is mainly related to time series matching techniques and document image processing.

Wafa KHLIF : PhD Student

Short-bio

Wafa KHLIF is currently a Second-year PhD student. She is co-supervised by Professor Jean-Christophe Burie at L3i Laboratory, University of La Rochelle (France) and Professor Adel Alimi at Regim Lab, National School of Engineers of Sfax (Tunisia). Wafa received the engineering diploma in computer science from the Tunisian engineering university ENIS-SFAX within the exchange program Erasmus mundus with Central Nantes in December 2014. In addition, she received the M.Sc. degree in 2015 from Polytech Nice-Sophia, the University of Nice Sophia Antipolis.

Julien Maitre : PhD Student

Short-bio

Julien MAITRE is currently a Second-year PhD student at L3i, University of La Rochelle (France). His domain of work is "Detection of weak signals in weakly structured data masses" and he is supervised by Prof. Michel Menard, Dr. Guillaume Chiron and associate Prof. Alain Bouju. Currently, he is mainly working on combine application of Word Embedding and Topic Modeling for weak signals detection and multi-agent system for Text Mining.

Summary

This course covers the fundamentals of different deep learning architectures, which will be explained through three types of mainstream applications, to image processing, pattern recognition and computer vision. A range of network architectures will be reviewed in this session.
In this session we will address the following : what is computer vision ; feature extraction and classification ; why deep learning (engineered features vs learned features) ; importance of sensing, high performance computing, and big data for deep learning ; image understand and ultimate goal of computer vision. We will address : Artificial Neural Networks basics ; artificial neuron characteristics (activation function) ; types of architectures (feed-forward networks vs. recurrent networks) ; types of learning rules ; Feed-forward networks and their training : Single Layer Perceptron (SLP), Multi-layer Perceptron (MLP), and back-propagation.

Contents

  • Back Propagation Feed Forward Neural Network
    • What is Feed Forward Neural Network
    • How back Propagation Work
      • One numerical demonstration of back propagation
    • Gradient Descent and it’s Variants
      • Batch Gradient Descent
      • Stochastic Gradient Descent
      • Mini-Batch Gradient Descent

  • Some practical tricks about how to train a NN
    • Early Stopping
    • Back Propagation with Momentum
    • Learning Rate
    • Delta Rule
    • Non Lineraity
    • Normalization Layer
  • Dropout
  • L2 Regularization
  • Early Stopping
  • Drop Connect
  • Model Averaging and Ensemble
  • L1 Regularization
  • Elastic Net Regularization
  • Max-Norm Constraints
  • Word-2-Vec
    • What is Word2Vec ?
    • Models of Neural Word Embedding
    • Application of Word Vectors

Session 2 : Recurrent Neural Network (RNN)

31st May (Thrushday) 14:00 - 16:00 ; Room - 018 (Bâtiment Pascal)

Guillaume Chiron : Post-Doctoral Researcher

Short-bio

Guillaume CHINON(Post-doc in computer science in L3i / University of La Rochelle). His research interests are at the intersection of both "machine learning" and "complex system modeling". So far, he has focused on two main applications : 1) 3D target tracking on videos and 2) texts analysis at library scale.

Hai Nguyen Thi Tuyet  : PhD Student

Short-bio

Thi-Tuyet-Hai NGUYEN is a 2nd Second-year PhD student at L3i, University of La Rochelle. Her domain of work is "Post-OCR text correction" and she is supervised by Prof. Antoine Doucet, Prof. Muriel Visani and Dr. Mickaël Coustaty. Currently, she is mainnly working on application of Recurrent Neural Network for Post-OCR text correction.

Mickael COUSTATY  : Assistant Professor

Short-bio

Mickael COUSTATY is currently a Assiatant Professor in the L3i laboratory, University of La Rochelle, France. He got a PhD in Computer Science at the University of La Rochelle, supervised by Prof. Jean-Marc Ogier in 2011 on the analysis of historical documents images in order to propose a new complex indexing process for CBIR. His research interests include Structural Pattern Recognition, Document Image Analysis, Camera -Based Document Analysis and Recognition, Machine Learning, Computer Vision, Augmented Reality and Semantic analysis.

Francisco Cruz : Post-Doctoral Researcher

Short-bio

Francisco Cruz FERNANDEZ is currently a Post-Doc researcher in the L3i laboratory, University of La Rochelle, France.

Summary

This course covers the fundamentals of Recurrent Neural Networks. Recurrent Neural Networks (RNNs) are popular models that have shown great promise in many NLP tasks. During this lecture, we will explain how RNNs work, and how to implement them.

Contents

  • Bi-Directional RNN
    • Standard Back Propagation
    • VGGNet
    • Back Propagation through time
    • Problem of Vanishing / Exploding gradient
  • Long-Short Term Memory
  • Gated Recurrent Units
  • Reservoir Computing
    • Echo State Nets
    • Liquid State Nets
    • Deep Echo State Nets
  • Deep Recurrent Neural Network
  • Encoding-Decoding Networks
  • Sequence to Sequence Network
  • Reinforcement Learning
  • One Shot Learning

Session 3 : Convolutional Neural Network

14th June (Thrushday) 14:00 - 17:00 ; Room - 018 (Bâtiment Pascal)

Van Nhu Van Nguyen : Post-Doctoral Researcher

Short-bio

Van NGUYEN, Ph.D. in Computer Vision. My research interests lie in the areas of areas of Computer Vision and Deep Learning (and Statistical Learning) with applications to layout analysis, object detection/segmentation, CBIR. Currently, he is working on the project iiBD which aims at the comic book images analysis, from panel/balloon segmentation and face/character detection to panel structure analysis. During his previous projects, he has worked on a fully automatic (lecture) videos indexing system and image segmentation problems.

Zuheng Ming : Post-Doctoral Researcher

Short-bio

Zuheng MING is a Post-doctoral research fellow at the L3i, University of La Rochelle, France, working in the project MOBIDEM with Dr.Joseph, Dr.Muzzamil and Pr.Jean-Christoph Burie. Previously, he has spent 2 years as a post-doc rechercher at LABRI of University of Bordeaux. He has obtained his PhD from GIPSA lab from University of Grenoble, France. His research interests mainly span in face analysis, object detection in computer vision, and multimodal machine learning (auido-video recognition). Currently, his primary research topics is mainly supervised deep learning and machine intelligence.

Nibal NAYEF : Post-Doctoral Researcher

Short-bio

Nibal NAYEF works currently as a post-doctoral researcher at Valconum and L3i Laboratory at the University of La Rochelle, France. She works on quality assessment and enhancement of mobile captured documents, information spotting and text / image segmentation. Nayef has a Ph.D. in computer science (2012) from the technical university of Kaiserslautern in Germany. She was a member of the IUPR laboratory (Image Understanding and Pattern Recognition) there, where she finished her PhD thesis entitled “Geometric-based symbol spotting and retrieval in technical line drawings”. Her research interests are : analysis and retrieval of line drawings and their associated evaluation protocols, statistical feature grouping, machine learning for vis ion, geometric matching and document image quality assessment and enhancement. She is a regular reviewer in IJDAR journal and DAS, ICDAR, ICFHR, ICPR conferences .

Summary

This course will cover the foundations of deep learning with its application to vision and text understanding. Attendees will become familiar with the concepts of deep learning. The ideas will be given on Convolutional Neural Networks (CNN) and it’s architecture. Different famous architectures of convolutonal neural network will also be discussed such as AlexNet, VGGNet, ResNet etc. Moreover, various applicatino of CNN will also be discussed such as the application in Object Detetion, Semantic Analysis etc. Lectures will provide intuitions, the underlying mathematics, typical applications, code snippets and references. By the end of these lecture, attendees are expected to gain enough familiarity to be able to apply these basic tools to standard datasets on their own.

Contents

  • Introduction
  • Typical CNN layers
  • Convolutional layer
  • Max-pooling layer
  • Loss function
    • Soft max Loss
    • VM Hinge Loss
    • Euclidean Loss
  • Training CNN
  • Data Augmentation
  • Batch Normalization

  • Application of CNN
    • Image Classification
      • AlexNet
      • VGGNet
      • ResNet
      • GoogleNet
      • Fast R-CNN
  • Object Detection and Localization
  • Semantic Segmentation
  • Pose Estimation
  • Video Classification
  • Action Recognition
  • Scene Understanding
  • Tracing in Video
  • Pose Estimation

Session 4 : Auto Encoder

10th July (Tuesday) 14:00 - 16:30 ; Room - 018 (Bâtiment Pascal)

Tanmoy Mondal : Post-Doctoral Researcher

Short-bio

Tanmoy Mondal received B.Tech. degree in Information Technology from West Bengal University of Technology, Kolkata, India, in 2007 and the M.Tech. degree in Mechatronics & Robotics from Bengal Engineering and Science University, Kolkata, in 2009. Before joining as a PhD student at Université de Tours (France) in 2012, he worked at several industries and premier R&D centers as a researcher. After completing his PhD from Laboratoire d’Informatique, Université François Rabelais, Tours, France in 2015, currently he did Post-Doc at INSA, Lyon, France. Currently, he is working as Post-Doc at L3i, University of La Rochelle. His research interests include pattern recognition, image processing and analysis, and computer vision. His current research is mainly related to time series matching techniques and document image processing.

Joseph Chazalon  : Post-Doctoral Researcher

Short-bio

Joseph Chazalon received Master of Scince and Ph.D. degrees in Computer Science from the Institut National des Sciences Appliquées (INSA) in Rennes (France) in 2008 and 2013. He currently is a research engineer at the L3i laboratory at the University of La Rochelle, France. His major research interests include visual languages, documents image processing, mobile document image acquisition, content based image retrieval and performance evaluation.

Ahmed Hamdi : Post-Doctoral Researcher

Short-bio

Ahmed Hamdi is a post-doctoral researcher in the L3i lab (university of La Rochelle). He is working on Document Flow Segmentation since 2017. He is interested on many techniques of information retrieval, semantic similarity and machine learning. Ahmed did his PhD in Aix-Marseille University from 2012 to 2016. He has worked on Natural Language Processing and mainly on morphological and syntactical analysis and generation.

Summary

The process of learning is essential for building natural or artificial intelligent systems. Thus, not surprisingly, machine learning is at the center of artificial intelligence today. And deep learning—essentially learning in complex systems comprised of multiple processing stages—is at the forefront of machine learning. The lectures will provide an overview of the fundamentals of unsupervised learning. We will discuss about the different models of unsupervised learning. There are two principal category of unsupervised learning exists in the literature such as Auto-Encoder and Generative Adverserial Networks (GANs). We will talk about various models of Auto-Encoder as well as GANs, which has been vastly used in the literature for generating data such as images, texts etc. by training the model with the use of exiting unlabelled datasets.

Contents

  • Theory of Auto Encoder
    • Basics of auto encoder
    • Auto encoder Layers
    • Auto encoder Algorithm
    • Loss Function
    • Training Over and Under Representation
  • Denoising Auto encoder
  • Sparse Auto encoder
  • Stacked Auto encoder
  • Applications
  • De-convolution Network
  • Doc-2-Vec
    • Context and Motivation
    • Models and Architecture
    • Applications of Doc-2-Vec

Session 5 : Hands-On Experience

12th July (Thrushday) 14:00 - 17:00 ; Room - 018 (Bâtiment Pascal)

Van Nhu Van Nguyen : Post-Doctoral Researcher

Short-bio

Van NGUYEN, Ph.D. in Computer Vision. My research interests lie in the areas of areas of Computer Vision and Deep Learning (and Statistical Learning) with applications to layout analysis, object detection/segmentation, CBIR. Currently, he is working on the project iiBD which aims at the comic book images analysis, from panel/balloon segmentation and face/character detection to panel structure analysis. During his previous projects, he has worked on a fully automatic (lecture) videos indexing system and image segmentation problems.

Zuheng Ming : Post-Doctoral Researcher

Short-bio

Zuheng MING is a Post-doctoral research fellow at the L3i, University of La Rochelle, France, working in the project MOBIDEM with Dr.Joseph, Dr.Muzzamil and Pr.Jean-Christoph Burie. Previously, he has spent 2 years as a post-doc rechercher at LABRI of University of Bordeaux. He has obtained his PhD from GIPSA lab from University of Grenoble, France. His research interests mainly span in face analysis, object detection in computer vision, and multimodal machine learning (auido-video recognition). Currently, his primary research topics is mainly supervised deep learning and machine intelligence.

Joseph Chazalon  : Post-Doctoral Researcher

Short-bio

Joseph Chazalon received Master of Scince and Ph.D. degrees in Computer Science from the Institut National des Sciences Appliquées (INSA) in Rennes (France) in 2008 and 2013. He currently is a research engineer at the L3i laboratory at the University of La Rochelle, France. His major research interests include visual languages, documents image processing, mobile document image acquisition, content based image retrieval and performance evaluation.

Summary

In this session we will talk about how to use different deel learning models, which exists in existing libraries such as TensorFlow, Keras etc. We will start this session by giving a brief overview of Python Language and then we will explain how to use TensorFlow and Keras library to use the models described in previous sessions. Moreover, we will also show some toy examples by using Python and would show how to implement some simple deeplearning models by using the available deep learning blocks exists.

Contents

  • Small Talk on Python Language
    • Tools needed to be installed
    • Some toy Examples on Python Programming
  • Small talk on TensorFlow
    • Installation
    • Some toy examples using Tensor Flow
    • How to use aforementioned models using TensorFlow
  • Detailed talk on Keras
    • Installation
    • Some toy examples using Keras
    • How to use aforementioned models using Keras
  • Practical Examples (Achived by TensroFlow)
    • Image classification
    • Object Detection

publie le mardi 8 mai 2018