Computer Vision and Deep Learning are increasingly establishing their connection into our daily lives, from consumers to businesses, but their future direction isn’t exactly clear. Having access to a one-day learning intensive with the industry’s corporate and startup pioneering research minds is exactly what IMVC 2019 provided this year and the overall sentiment paints an exciting future for video and image processing along with deep learning developments.

The keynotes and technical lectures dealt with the forefront of computer vision innovations, and in light of the wealth of information shared, we decided to summarize some noteworthy presentations and their key insights.

Here’s a summary of the startups and industry speaker presentations, and academic AI research papers and competition winners:

SPEAKERS

 

Briefcam
Shmuel Peleg, Co-founder and Chief Scientist

Looking at video affects your sound perception, and it’s a tricky problem to solve when viewing many people speaking. There’s plenty of video recognition research breaking barriers but seldom does it tackle audio too. Vid2speech is Briefcam’s new project that aims to accurately isolate the audio and video cues based on a dilated convolutional neural network in an audio-visual deep model architecture. The technology enhances the speech of the subject when using a video that shows the face of the speaker(s). It can be applied to isolating  the audio of spoken video. Professor Peleg demonstrated the technology on ESPN’s sports talk show First Take with their notorious personalities Stephen A Smith and Max Kellerman, who commonly talk on top of each other, resulting in incoherent content. Peleg demonstrated Vid2speech’s ability to mute the audio of one of the commentators while both were yelling simultaneously.

 

IBM
Aya Soffer, VP of AI Tech

Aya Soffer stated audio and image recognition has been conquered by AI computers, surpassing the skill of humans. But the eclipse hasn’t happened for video recognition yet. Still, human intuition can identify the content of a video better than the best deep learning algorithms. Video comprehension is a major focus for IBM’s research team directed at enterprise applications. Their team has created a new mega video data set called Moments In Time, in collaboration with MIT University, comprised of millions of video clips to support their development efforts, such as segmentation of video into semantic scenes, interpolating full videos scenes from select frames, recognizing objects and humans actions, summarized highlights of a video and few shot learning for object recognition.

 

Blink Technologies
Soliman Nasser, Lead Research and Development

Nasser began to explain the geometry of the human eye before detailing their technology for eye tracking. They track the eye movements from RGB cameras in order to predict the gaze point and consequently gaze vectors, the connection between your eye point and gaze point. They reconstruct a 3D model of the eye to find the intrinsic parameters of the eye and define the optical axis of each eye, and consequently calculate the point of gaze using triangulation methods. They crowd source their data (real and synthetic) and feed it into a deep learning algorithm to predict the gaze vector. The market is largely saturated with intrusive infrared sensors whereas Blink Technologies’ algorithms works on any RGB camera in real world environments with limited CPU computation power and without GPU units.

 

Lightricks
Ofir Bibi, Director of Research

According to Bibi, around 50% of the pixels in the typical image are composed of the sky, and in order to achieve the best capture, Lightricks believes the sky portion should be aesthetically perfect. The startup’s research team trained a neural network to segment the sky background on still images, that’s integrate into their highly popular Quickshot application. Video, similarly poses a greater challenge and the startup developed an algorithm for automatic replacement of the sky portion of video clips on composited videos with enhanced skys. They achieve temporal consistency in the segmentation network by using a complex feedback loop and novel infrastructure and their algorithm trains by actually adding noise to the data in all kinds of areas to extend and dilate the boundaries that mimic the errors of real video. Their work was conducted by Tavi Halperin, Harel Cain and Michael Werman and will be presented at Eurographics 2019.

 

Imagry
Adham Ghazali, co-founder and CEO

Autonomous driving software is a complex problem. Depth is the major drawback for cameras, and LiDar doesn’t have high resolution information on active perception. Even the leading contender in the Autonomous Driving race, Waymo, has made advances in the localization by creating high definition maps and human labor (1 hour of self driving = 800 hours of annotation to build a high definition map) and SLAM for optimization. But if a map changes, they need to redo the procedure again which poses a problem to scaling. Imagry developed a novel architecture for autonomous driving software that’s able to localize, perceive and drive in unknown environment – learning a driving policy without a heuristic. It’s capable of training on one city’s roads, and drive autonomously in another city. The startup also invented a motion path finding algorithm called Aleph* (open sourced), which takes two hours to train on a laptop for a new function, exhibited promising run time efficiency, and compared well to industry benchmark Nstep Duke UN. Mangan encourage the industry to conduct a comparison to Monte Carlo research.

 

VayaVision
Shmoolik Mangan, Algorithms Development Manager

VayaVision developed a low level raw data fusion and perception system for autonomous vehicles, as opposed to object fusion architecture. They use unified algorithms – unsupervised neural networks – to detect objects based on the perceived environment from all pixel level data from RGBd sensor data. Deep learning and computer vision algorithms are applied to the upsampled HD 3D model. While the typical sensor set for autonomous vehicles consists of several sensor types, upon sensor failure, the startup’s novel redundancy mechanisms allow the vehicle to continue driving and not become stranded on the active road, albeit continue at reduced speeds and only to the nearest vehicle garage.

 

DeePathology.ai
Jacob Gildenblat, Co-founder and CTO

Automating cell detection requires annotating large amounts of data, which is usually very unbalance and to alleviate the arduous task, DeePathology.ai developed the Cell Detection Studio, a DIY tool for pathologists develop their own AI solutions to train deep learning cell detection algorithms on their own data. Using this tool, Pathologists can easily create and train deep learning cell detection algorithms off of their own data sets, with the help of Active Learning.

 

Google
Yael Pritch, Leader of Goole Perception AI

The Google team is invested in putting computational photography and Artificial Intelligence  inside mobile cameras. According to Pritch, software advancements are the current innovation in the mobile camera today, opposed to the optical hardware that’s constricted by the physical form factors.  Google’s innovation in two new camera upgrades were on display – portrait mode and night sight – detailing the convolutional neural networks and phase different auto-focus that’s unanimously won over conventional camera critics. Night sight is a feature for the Pixel 1,2,3, which gives you a high quality capture in a low-level light environment. The portrait mode makes use of a machine learning network that learns to segment people from images and a mechanism that utilizes another lens pixel for autofocus which is repurposed to create a high quality depth map. The Google Pixel 2 computational camera won the DPReview Awards in 2017, and the Google Pixel 3 won the same award in 2018. It’s the leading camera today and fueled by the AI software. Everything is running on the device in thermal and memory limitations on the Pixel 1 2 and 3, and the Pixel software is back compatible to all models.

 

STUDENT COMPETITION WINNERS

 

Graduate: 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
By: Itzik Ben Shabat
Solving the classification problem: with the abundance of RGB cameras, Ben-Shabat sought to solve the 3D point cloud with deep learning methods, even though the power of Convolutional Neural Networks yields different results on point clouds as they do on images due to the inherent challenges of structure, order and sample size. To use the power of CNNs, they proposed a new hybrid representation which relies on point statistics.

The work was also extended to localized geometry which was accepted to CVPR 2019.

 

Undergraduate: Beholder-GAN: Generation and Beautification of Facial Images with Conditioning on Their Beauty Level
By: Dean Zadok, Nir Diamant, Chaim Baskin, and Eli Schwartz

The group trained a generative model that generates faces according to a measurement of beauty using their unsupervised beautification model.

 

STUDENT RESEARCH PAPERS

 

Spoofing in Face Recognition Systems Based on Projective Invariants and Stereo Recording
Alexander Naitsat, Technion

On Occlusion Removal in Ultrasound Imaging
Yossef Cohen, Technion
Lung Structure Enhancement in Chest Radiographs via CT based FCNN Trainning
Ophir Gozes, Tel Aviv University

R(D)-Constrained Adaptive Quantization for Coding and De-Noising of Medical Images
Shira Nemirovsky-Rotman, Technion

xUnit: Learning a Spatial Activation Function for Efficient Image Restoration
Idan Kligvasser, Technion

SOS Boosting for Image Deblurring Algorithms
Shahar Romem Peled, Technion

Spatio-Temporal Detection of Cumulonimbus Clouds in Infrared Satellite Images
Ron Dorfman and Etai Wagner, Technion

3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
Itzik Ben Shabat, Technion

From Unable to Enable, How Multiplicative Beamforming Can Enable Advanced Image Processing Techniques in Portable Ultrasound Devices
Omri Soceanu, Technion

Early Detection of Cancer Using Thermal Video Analysis
Idun Barazani and Ori Bryt, Technion

Automated Assembly of Polygonal Jigsaw Puzzles
Peleg Harel, Ben Gurion University

Modeling the Retinal Cone Mosaic and Its Use for Individual Authentication
Keren Berger, Ben Gurion University

On Sweeping Patterns Classification
Avner Atias, Technion

On the Role of Geometry in Geo-Localization
Moti kadosh, IDC Herzliya

Explorations and Lessons Learnt in Building an Autonomous Formula Sae Car From Simulations
Dean Zadok, Amir Biran and Tom Hirshberg, Technion

Network Adaptation Strategies for Learning New Classes Without Forgetting the Original Ones
Hagai Taitelbaum, Bar Ilan University

Autonomous Aero Optics Using Active flow Control for Vision Sensor Cleaning
David Menicovich, Actasys Inc.

Decision Trees Based Image Segmentation Using Ensemble Clustering
Yaron Levi, Yezreel Valley College

Beholder-GAN: Generation and Beautification of Facial Images with Conditioning on Their Beauty Level
Eli Schwartz and Chaim Baskin, Technion

LaSO: Label-Set Operations Networks for Multi-Label Few-Shot Learning
Amit Alfassy, Technion

Domain-Aware Generalized Zero-Shot Learning
Yuval Atzmon, Bar Ilan University

Form a Deep Learning Model Back to The Brain – Inferring Morphological Markers and Their Relation to Aging
Gidon Levakov, Ben Gurion University

Counting Tomato Flowers from Video using Deep Learning
Simo Hanouna and Stav Lutzky, Ben Gurion University

Fractal Features and Local Phase Information in Textures Recognition
Samah Khawaled, Technion

Microscopy Cell Segmentation via Convolutional LSTM Networks
Assaf Arbelle, Ben Gurion University

Accelerated Magnetic Resonance Imaging by Generative Adversarial Neural Network
Roy Shaul and Itamar David, Ben Gurion University

Flexible DOF
Yuval Schwartz and Maayan Weitzman, Technion

 

STUDENT DEMOS

 

ARCADE – Accurate and Rapid Camera for Depth Estimation
Shay Elmalem and Yotam Gil, Tel Aviv University

Obtaining Physical Laws Empirically with Manifold Learning
Gal Kinberg, Kobi Shiran, Or Yair and Avi Rosen, Technion

Dance Dance Revolution
Avrham Aton, Yael Ben-Gigi, Mor Avi-Aharon and Nadav Amram, Ben Gurion University

Dazy Your Interactive Home Assistant
Dvir Kenig, Yuval Yoskovits, Ziv Levi and Arik Rotem, Ben Gurion University

Air GunZ
Ran Wiesman, Tom Shlomo, Itamar Beeri and Shahar Berger, Ben Gurion University

The IMVC conference is run by SagivTech (Chen and Nizan Sagiv) and Dr. Jacob Cohen and will take place next year at the same location.