Increasing Vision Performance by Preprocessing on the Frame Grabber

Product / 08.2019


Today’s cutting edge machine vision and video monitoring applications are taking on much more difficult requirements than ever before, such as identifying a small number of cancer cells in a patient’s blood and tracking which items customers leave a store with so their accounts can be charged without having to check out. Camera manufacturers are addressing these applications with sensors that offer dramatic increases in image quality and frames per second. Vision interface standards have kept pace by offering higher connection bit rates such the new CoaXPress 2.0 standard which offers data rates up to 12.5 Gigabits per second (Gbps).
These advances have strained central processing units (CPUs) that are now tasked with the need to process dramatically higher volumes of imaging data. Some applications are using high-end CPUs or dividing processing among several personal computers but both of these approaches are expensive and the latter also adds considerable size and weight. The result is that many applications are operating at CPU loads just under 100%, limiting their ability to deliver further performance gains.

Euresys has addressed these challenges by enabling camera Coaxlink-octo-and-CXP-12.pngmanufacturers and vision integrators to upload code to the FPGA in the company’s Coaxlink Octo and Coaxlink Quad CXP-12 frame grabbers to handle image processing tasks that would otherwise have to be performed on the host. The Euresys CustomLogic FPGA design kit can handle virtually any repetitive image preprocessing task that is performed on every pixel of the image. Typical applications include transforming the image based on a lookup table such as by converting the color from the RGB to the YUV scale, implementing a noise reduction algorithm, and compensating for sensor defects such as black pixels. Another common application is flat field correction, compensating for differences in light intensity over the field of view.

Performing image processing on the frame grabber provides dramatic improvements in image quality and processing speed, particularly for cutting-edge vision systems that are currently bottlenecked by the processing power of the host computer. The FPGA processes the image in parallel with image transfer so processing time savings on the host computer are achieved without adding any delays in the frame grabber. This means that vision systems can deliver higher resolution, higher image speeds and lower latency without increasing the cost of the host computer.
In the past, frame grabber suppliers have offered to incorporate their customers’ code into their FPGAs at the heart of their frame grabbers. This approach requires that users share their proprietary intellectual property so that it could be designed into the frame grabber.
 
In the new CustomLogic approach, on the other hand, users create and compile their own FPGA code into an object file and then upload it to the FPGA using a tool provided by the frame grabber company.
The vision integrator’s proprietary code never leaves its premises and the resulting FPGA would be extremely difficult and expensive to reverse engineer.
Many large machine makers have developed their own frame grabbers based on FPGAs that also perform image processing and analysis tasks. These frame grabbers in virtually every case use the venerable Camera Link protocol. The machine makers that designed them are now faced with the difficult challenge of redesigning the frame grabbers to accommodate the current generation of high-speed interfaces such as CXP-12. Their task can be greatly simplified by taking advantage of the built-in CXP interface and ability to upload their proprietary image processing routines offered by the new Euresys frame grabbers.

The CustomLogic FPGA design leaves up to 70% of the resources on the Xilinx Kintex Ultrascale XCKU035 FPGAs used in the Coaxlink Octo and Coaxlink Quad CXP-12 frame grabbers for image preprocessing.

FPGA-scale-(2).png

The design kit provides access to the CoaXPress camera pixel stream, on-board DDR4 memory and PCIe Gen3 connectivity. The design phase uses Xilinx Vivado development tools.

The Coaxlink CustomLogic design kit is delivered with a reference design consisting of a Xilinx Vivado project that exposes all interfaces available to the user.

ReferenceDesignCustomLogic-(1).png
 













The Data Stream (pixel) interface is based on the AMBA AXI4-Stream protocol. On the source side, this interface provides images acquired from the camera. The interface transfers the after processing by the user logic to the PCI Express DMA Back-End channel. The Control/Status interface allows the user to read and write registers inside the user logic via the Coaxlink Driver API. 
 
CustomLogic-(1).png

The event interface allows the user logic to send time-stamped events to the Memento Logging tool with a precision of 1 μs. Memento provides the developer with a precise timeline of time-stamped events, along with context information and logic analyzer view. It provides valuable assistance during application development and debugging, as well as during machine operation.

Vivado® High-Level Synthesis (HLS), included as a no cost upgrade in all Vivado HLx Editions, accelerates IP creation by enabling C, C++ and System C specifications to be directly targeted into Xilinx programmable devices without the need to manually create register transfer level (RTL) code. In addition to C++ programming skills, FPGA design requires understanding the constraints and limitations of FPGAs. For example, random access memory (RAM) limitations in FPGAs usually rule out memory allocation.

VivadoProjectCustomLogic_Interface1.png

VivadoProjectCustomLogic_VHD.png

CustomLogic can parallelize many image processing tasks, processing pixels on the fly without any buffering or latency. By offloading repetitive and massively parallel tasks to the FPGA, the CPU is freed to focus on high level tasks to meet the requirements of today’s cutting-edge imaging applications. The intellectual property included with CustomLogic such as the CoaXPress interface and the Memento log system helps get products to market in less time.