Laboratoire Informatique, Image et Interaction (L3i)

Seminar-DeepLearning-2018

Session 1 : Basic Neural Networks
17^th May (Thrushday) 14:00 - 17:00 ; Room - 018 (Bâtiment Pascal)
	Tanmoy Mondal : Post-Doctoral Researcher Short-bio Tanmoy Mondal received B.Tech. degree in Information Technology from West Bengal University of Technology, Kolkata, India, in 2007 and the M.Tech. degree in Mechatronics & Robotics from Bengal Engineering and Science University, Kolkata, in 2009. Before joining as a PhD student at Université de Tours (France) in 2012, he worked at several industries and premier R&D centers as a researcher. After completing his PhD from Laboratoire d’Informatique, Université François Rabelais, Tours, France in 2015, currently he did Post-Doc at INSA, Lyon, France. Currently, he is working as Post-Doc at L3i, University of La Rochelle. His research interests include pattern recognition, image processing and analysis, and computer vision. His current research is mainly related to time series matching techniques and document image processing.
	Wafa KHLIF : PhD Student Short-bio Wafa KHLIF is currently a Second-year PhD student. She is co-supervised by Professor Jean-Christophe Burie at L3i Laboratory, University of La Rochelle (France) and Professor Adel Alimi at Regim Lab, National School of Engineers of Sfax (Tunisia). Wafa received the engineering diploma in computer science from the Tunisian engineering university ENIS-SFAX within the exchange program Erasmus mundus with Central Nantes in December 2014. In addition, she received the M.Sc. degree in 2015 from Polytech Nice-Sophia, the University of Nice Sophia Antipolis.
	Julien Maitre : PhD Student Short-bio Julien MAITRE is currently a Second-year PhD student at L3i, University of La Rochelle (France). His domain of work is "Detection of weak signals in weakly structured data masses" and he is supervised by Prof. Michel Menard, Dr. Guillaume Chiron and associate Prof. Alain Bouju. Currently, he is mainly working on combine application of Word Embedding and Topic Modeling for weak signals detection and multi-agent system for Text Mining.
Summary This course covers the fundamentals of different deep learning architectures, which will be explained through three types of mainstream applications, to image processing, pattern recognition and computer vision. A range of network architectures will be reviewed in this session. In this session we will address the following : what is computer vision ; feature extraction and classification ; why deep learning (engineered features vs learned features) ; importance of sensing, high performance computing, and big data for deep learning ; image understand and ultimate goal of computer vision. We will address : Artificial Neural Networks basics ; artificial neuron characteristics (activation function) ; types of architectures (feed-forward networks vs. recurrent networks) ; types of learning rules ; Feed-forward networks and their training : Single Layer Perceptron (SLP), Multi-layer Perceptron (MLP), and back-propagation.

Contents

Back Propagation Feed Forward Neural Network

What is Feed Forward Neural Network

How back Propagation Work

One numerical demonstration of back propagation

Gradient Descent and it’s Variants

Batch Gradient Descent

Stochastic Gradient Descent

Mini-Batch Gradient Descent

Some practical tricks about how to train a NN

Early Stopping

Back Propagation with Momentum

Learning Rate

Delta Rule

Non Lineraity

Normalization Layer

Dropout

L2 Regularization

Early Stopping

Drop Connect

Model Averaging and Ensemble

L1 Regularization

Elastic Net Regularization

Max-Norm Constraints

Word-2-Vec

What is Word2Vec ?

Models of Neural Word Embedding

Application of Word Vectors

Session 2 : Recurrent Neural Network (RNN)
31^st May (Thrushday) 14:00 - 16:00 ; Room - 018 (Bâtiment Pascal)
	Guillaume Chiron : Post-Doctoral Researcher Short-bio Guillaume CHINON(Post-doc in computer science in L3i / University of La Rochelle). His research interests are at the intersection of both "machine learning" and "complex system modeling". So far, he has focused on two main applications : 1) 3D target tracking on videos and 2) texts analysis at library scale.
	Hai Nguyen Thi Tuyet : PhD Student Short-bio Thi-Tuyet-Hai NGUYEN is a 2nd Second-year PhD student at L3i, University of La Rochelle. Her domain of work is "Post-OCR text correction" and she is supervised by Prof. Antoine Doucet, Prof. Muriel Visani and Dr. Mickaël Coustaty. Currently, she is mainnly working on application of Recurrent Neural Network for Post-OCR text correction.
	Mickael COUSTATY : Assistant Professor Short-bio Mickael COUSTATY is currently a Assiatant Professor in the L3i laboratory, University of La Rochelle, France. He got a PhD in Computer Science at the University of La Rochelle, supervised by Prof. Jean-Marc Ogier in 2011 on the analysis of historical documents images in order to propose a new complex indexing process for CBIR. His research interests include Structural Pattern Recognition, Document Image Analysis, Camera -Based Document Analysis and Recognition, Machine Learning, Computer Vision, Augmented Reality and Semantic analysis.
	Francisco Cruz : Post-Doctoral Researcher Short-bio Francisco Cruz FERNANDEZ is currently a Post-Doc researcher in the L3i laboratory, University of La Rochelle, France.
Summary This course covers the fundamentals of Recurrent Neural Networks. Recurrent Neural Networks (RNNs) are popular models that have shown great promise in many NLP tasks. During this lecture, we will explain how RNNs work, and how to implement them. Contents Bi-Directional RNN Standard Back Propagation VGGNet Back Propagation through time Problem of Vanishing / Exploding gradient Long-Short Term Memory Gated Recurrent Units Reservoir Computing Echo State Nets Liquid State Nets Deep Echo State Nets Deep Recurrent Neural Network Encoding-Decoding Networks Sequence to Sequence Network Reinforcement Learning One Shot Learning

Session 3 : Convolutional Neural Network
14^th June (Thrushday) 14:00 - 17:00 ; Room - 018 (Bâtiment Pascal)
	Van Nhu Van Nguyen : Post-Doctoral Researcher Short-bio Van NGUYEN, Ph.D. in Computer Vision. My research interests lie in the areas of areas of Computer Vision and Deep Learning (and Statistical Learning) with applications to layout analysis, object detection/segmentation, CBIR. Currently, he is working on the project iiBD which aims at the comic book images analysis, from panel/balloon segmentation and face/character detection to panel structure analysis. During his previous projects, he has worked on a fully automatic (lecture) videos indexing system and image segmentation problems.
	Zuheng Ming : Post-Doctoral Researcher Short-bio Zuheng MING is a Post-doctoral research fellow at the L3i, University of La Rochelle, France, working in the project MOBIDEM with Dr.Joseph, Dr.Muzzamil and Pr.Jean-Christoph Burie. Previously, he has spent 2 years as a post-doc rechercher at LABRI of University of Bordeaux. He has obtained his PhD from GIPSA lab from University of Grenoble, France. His research interests mainly span in face analysis, object detection in computer vision, and multimodal machine learning (auido-video recognition). Currently, his primary research topics is mainly supervised deep learning and machine intelligence.
	Nibal NAYEF : Post-Doctoral Researcher Short-bio Nibal NAYEF works currently as a post-doctoral researcher at Valconum and L3i Laboratory at the University of La Rochelle, France. She works on quality assessment and enhancement of mobile captured documents, information spotting and text / image segmentation. Nayef has a Ph.D. in computer science (2012) from the technical university of Kaiserslautern in Germany. She was a member of the IUPR laboratory (Image Understanding and Pattern Recognition) there, where she finished her PhD thesis entitled “Geometric-based symbol spotting and retrieval in technical line drawings”. Her research interests are : analysis and retrieval of line drawings and their associated evaluation protocols, statistical feature grouping, machine learning for vis ion, geometric matching and document image quality assessment and enhancement. She is a regular reviewer in IJDAR journal and DAS, ICDAR, ICFHR, ICPR conferences .
Summary This course will cover the foundations of deep learning with its application to vision and text understanding. Attendees will become familiar with the concepts of deep learning. The ideas will be given on Convolutional Neural Networks (CNN) and it’s architecture. Different famous architectures of convolutonal neural network will also be discussed such as AlexNet, VGGNet, ResNet etc. Moreover, various applicatino of CNN will also be discussed such as the application in Object Detetion, Semantic Analysis etc. Lectures will provide intuitions, the underlying mathematics, typical applications, code snippets and references. By the end of these lecture, attendees are expected to gain enough familiarity to be able to apply these basic tools to standard datasets on their own.

Contents
Introduction Typical CNN layers Convolutional layer Max-pooling layer Loss function Soft max Loss VM Hinge Loss Euclidean Loss Training CNN Data Augmentation Batch Normalization	Application of CNN Image Classification AlexNet VGGNet ResNet GoogleNet Fast R-CNN Object Detection and Localization Semantic Segmentation Pose Estimation Video Classification Action Recognition Scene Understanding Tracing in Video Pose Estimation

Session 4 : Auto Encoder
10^th July (Tuesday) 14:00 - 16:30 ; Room - 018 (Bâtiment Pascal)
	Tanmoy Mondal : Post-Doctoral Researcher Short-bio Tanmoy Mondal received B.Tech. degree in Information Technology from West Bengal University of Technology, Kolkata, India, in 2007 and the M.Tech. degree in Mechatronics & Robotics from Bengal Engineering and Science University, Kolkata, in 2009. Before joining as a PhD student at Université de Tours (France) in 2012, he worked at several industries and premier R&D centers as a researcher. After completing his PhD from Laboratoire d’Informatique, Université François Rabelais, Tours, France in 2015, currently he did Post-Doc at INSA, Lyon, France. Currently, he is working as Post-Doc at L3i, University of La Rochelle. His research interests include pattern recognition, image processing and analysis, and computer vision. His current research is mainly related to time series matching techniques and document image processing.
	Joseph Chazalon : Post-Doctoral Researcher Short-bio Joseph Chazalon received Master of Scince and Ph.D. degrees in Computer Science from the Institut National des Sciences Appliquées (INSA) in Rennes (France) in 2008 and 2013. He currently is a research engineer at the L3i laboratory at the University of La Rochelle, France. His major research interests include visual languages, documents image processing, mobile document image acquisition, content based image retrieval and performance evaluation.
	Ahmed Hamdi : Post-Doctoral Researcher Short-bio Ahmed Hamdi is a post-doctoral researcher in the L3i lab (university of La Rochelle). He is working on Document Flow Segmentation since 2017. He is interested on many techniques of information retrieval, semantic similarity and machine learning. Ahmed did his PhD in Aix-Marseille University from 2012 to 2016. He has worked on Natural Language Processing and mainly on morphological and syntactical analysis and generation.
Summary The process of learning is essential for building natural or artificial intelligent systems. Thus, not surprisingly, machine learning is at the center of artificial intelligence today. And deep learning—essentially learning in complex systems comprised of multiple processing stages—is at the forefront of machine learning. The lectures will provide an overview of the fundamentals of unsupervised learning. We will discuss about the different models of unsupervised learning. There are two principal category of unsupervised learning exists in the literature such as Auto-Encoder and Generative Adverserial Networks (GANs). We will talk about various models of Auto-Encoder as well as GANs, which has been vastly used in the literature for generating data such as images, texts etc. by training the model with the use of exiting unlabelled datasets. Contents Theory of Auto Encoder Basics of auto encoder Auto encoder Layers Auto encoder Algorithm Loss Function Training Over and Under Representation Denoising Auto encoder Sparse Auto encoder Stacked Auto encoder Applications De-convolution Network Doc-2-Vec Context and Motivation Models and Architecture Applications of Doc-2-Vec

Session 5 : Hands-On Experience
12^th July (Thrushday) 14:00 - 17:00 ; Room - 018 (Bâtiment Pascal)
	Van Nhu Van Nguyen : Post-Doctoral Researcher Short-bio Van NGUYEN, Ph.D. in Computer Vision. My research interests lie in the areas of areas of Computer Vision and Deep Learning (and Statistical Learning) with applications to layout analysis, object detection/segmentation, CBIR. Currently, he is working on the project iiBD which aims at the comic book images analysis, from panel/balloon segmentation and face/character detection to panel structure analysis. During his previous projects, he has worked on a fully automatic (lecture) videos indexing system and image segmentation problems.
	Zuheng Ming : Post-Doctoral Researcher Short-bio Zuheng MING is a Post-doctoral research fellow at the L3i, University of La Rochelle, France, working in the project MOBIDEM with Dr.Joseph, Dr.Muzzamil and Pr.Jean-Christoph Burie. Previously, he has spent 2 years as a post-doc rechercher at LABRI of University of Bordeaux. He has obtained his PhD from GIPSA lab from University of Grenoble, France. His research interests mainly span in face analysis, object detection in computer vision, and multimodal machine learning (auido-video recognition). Currently, his primary research topics is mainly supervised deep learning and machine intelligence.
	Joseph Chazalon : Post-Doctoral Researcher Short-bio Joseph Chazalon received Master of Scince and Ph.D. degrees in Computer Science from the Institut National des Sciences Appliquées (INSA) in Rennes (France) in 2008 and 2013. He currently is a research engineer at the L3i laboratory at the University of La Rochelle, France. His major research interests include visual languages, documents image processing, mobile document image acquisition, content based image retrieval and performance evaluation.
Summary In this session we will talk about how to use different deel learning models, which exists in existing libraries such as TensorFlow, Keras etc. We will start this session by giving a brief overview of Python Language and then we will explain how to use TensorFlow and Keras library to use the models described in previous sessions. Moreover, we will also show some toy examples by using Python and would show how to implement some simple deeplearning models by using the available deep learning blocks exists. Contents Small Talk on Python Language Tools needed to be installed Some toy Examples on Python Programming Small talk on TensorFlow Installation Some toy examples using Tensor Flow How to use aforementioned models using TensorFlow Detailed talk on Keras Installation Some toy examples using Keras How to use aforementioned models using Keras Practical Examples (Achived by TensroFlow) Image classification Object Detection

publie le mardi 8 mai 2018