Today’s camera-equipped IoT gadgets such as smart displays, VR, sport cams, security cameras, and robotics become a lot more useful when they not only capture imagery but interpret and act on what they see. Artificial Intelligence (AI) is an emerging tech already enabling cameras with object detection, tracking, classification and facial recognition. Further, because the camera itself has the intelligence, it can decide how to respond based on what it knows instead of waiting for video data to be sent to the cloud and analyzed. This is what’s called “edge computing” that helps get past challenges around massive data generation and latency in camera applications.
Enabling developers to bring these features to cameras, Chipmaker Qualcomm has unveiled two new vision chips powering its “Vision Intelligence” Platform for applications including commercial and home surveillance cameras, VR 360-degree cameras, robots, smart displays, sports cameras, wearable cameras, and more.
The new 10nm FinFET SoCs, the QCS605 and QCS603, are said to support powerful computing for on-device camera processing and machine learning, with exceptional power and thermal efficiency.
The chips feature the firm’s advanced image signal processor (ISP) and Qualcomm Artificial Intelligence (AI) Engine, ARM-based multicore CPU, eight Qualcomm Kryo 360 CPU cores, Hexagon 685 Vector Processor, and Adreno 615 GPU. It also includes Qualcomm’s advanced camera processing software, machine learning and computer vision software development kits (SDKs), connectivity and security technologies.
The platform supports up to 4K video resolution at 60 frames per second (fps), or 5.7K at 30 fps. It features a dual 14-bit Qualcomm Spectra 270 ISP that supports dual 16 megapixels sensors.
Also included are advanced vision processing capabilities necessary for IoT applications such as staggered HDR, advanced electronic image stabilization, de-warp, de-noise, chromatic aberration correction, and motion compensated temporal filters in hardware.
For AI and neural processing, manufacturers can use frameworks like Tensorflow, Caffe and Caffe2, as well as the Open Neural Network Exchange format.
Qualcomm also notified that KEDACOM and Ricoh THETA are planning to utilize its Vision Intelligence Platform in their products.
Currently, the Qualcomm QCS605 and QCS603 SoCs are sampling with multiple SKUs that are designed to meet a variety of technical and cost-effectiveness requirements.
The QCS605-based VR 360 camera’s reference designs from Qualcomm Technologies and Altek Corporation are available today, and QCS603-based industrial security camera reference will be available in the second half of 2018.