What Is a Camera ISP, and What Does It Actually Do?
Connect a bare image sensor to your host, capture a frame, and open it. What you get is a flat, grey, oddly colored grid that looks nothing like the scene in front of the lens. The sensor did its job. Nothing has turned its output into an image yet.
That job belongs to the camera ISP. A camera image signal processor takes the sensor’s raw measurements and runs them through a fixed sequence of correction stages until a viewable frame comes out the other end. An ISP camera does that work inside the module, so your host receives finished video instead of sensor data.
This post answers what an ISP is in a camera, walks through every stage of the pipeline, explains the onboard ISP vs host ISP decision, and lists which Vadzo modules carry a camera ISP and which hand you raw Bayer frames instead.
What is a camera ISP, and why is it required for raw sensors?
A camera image signal processor is the block that converts raw sensor data into a color-correct, viewable image.
An image sensor measures light intensity, not color. Color comes from a filter grid bonded over the pixel array, so each pixel records only red, green, or blue. Everything else has to be reconstructed.
The camera ISP does that reconstruction, then corrects the physics of the optical path on top of it. Without a camera ISP you have measurements. With one you have an image.
Image Signal Processor Explained: What the Block Actually Does
Here is the image signal processor explained in one line: it is a fixed-order pipeline that repairs, reconstructs and formats sensor output.
A Bayer pattern sensor produces a single-channel mosaic. Each pixel of a Bayer pattern sensor holds one color value, arranged in a repeating red-green-green-blue grid across the array.
Vadzo’s raw-output cameras make this concrete. The Bolt-2020BRS and Falcon-2020BRS both stream 8-bit Bayer raw from the Onsemi AR2020 sensor with no demosaic applied, so your host receives exactly the mosaic the sensor produced.
That is useful if you want control. It is a problem if you expected a picture. This is the whole reason an ISP camera exists as a product category.
Camera ISP Pipeline Stages: From Bayer Raw to YUV
The ISP pipeline runs in a fixed order because each stage depends on the one before it. Read it as a dependency chain.
Black level correction comes first. Sensors report a small non-zero value even in complete darkness, and that pedestal must be subtracted before anything else, because every later stage inherits the offset if it is skipped or mis-set.
Lens shading correction follows, compensating for brightness and color falloff toward the corners of the frame. It is specific to the optical design, so it needs per-design calibration rather than a generic profile.
Demosaicing is where the image appears. The demosaicing algorithm interpolates the two missing color values at every pixel from its neighbors, performing the RAW-to-RGB conversion. Its quality determines whether fine detail survives or turns into colored fringing.
Noise reduction comes next, removing noise that rises with gain, so it matters most in low light. This is the sharpest trade-off in the pipeline, because tuning it too hard removes real detail along with the noise.
The color correction matrix then maps the sensor's native response into a standard color space, and gamma correction encodes the result for display. A wrong color correction matrix makes skin tones, indicator LEDs and calibration targets all read incorrectly. Skip gamma correction and the frame looks unnaturally dark.
Output formatting closes the pipeline, and this is where the YUV vs RAW output decision becomes visible to software. Vadzo's Falcon and Merlin USB cameras deliver fully processed video at the camera output, so the host receives finished frames rather than sensor data, with every stage above already run before the frame leaves the module.
The 3A Algorithms: Auto Exposure, Auto White Balance, and Auto Focus
The 3A algorithms are control loops running alongside the pipeline rather than inside it.
Auto exposure control camera loops set integration time and gain from scene brightness. The auto white balance camera loop estimates the illuminant and corrects the color cast, so white stays white under tungsten, daylight or LED lighting. Auto focus drives the lens actuator.
Where those loops measure matters as much as what they measure. A full-frame auto exposure control camera loop averages the whole scene, which fails the moment your subject sits against a bright window.
That is what ROI auto-exposure camera control fixes. The Falcon-1335CRA supports ROI-based auto exposure and auto focus, so an ROI auto exposure camera loop responds to the region you care about rather than to the whole frame. On a backlit scene, ROI auto-exposure camera behavior is the difference between a usable frame and a silhouette.
The auto white balance camera loop has the same regional sensitivity. The Falcon-821CRS documents HDR alongside auto white balance and auto exposure, which is the combination most 3A algorithms implementations expose to the integrator.
HDR Modes: iHDR, LI-HDR, and eDR
High dynamic range is an ISP function, and an HDR camera module implements it differently depending on the sensor and the ISP behind it.
The Falcon-1335CRA provides iHDR, or integrated HDR. The Falcon-830CRS and Falcon-544CRS provide line-interleaved HDR along with eDR. The Falcon-821CRS provides HDR with auto white balance and auto exposure.
If your scene contains both direct sunlight and deep shadow, an HDR camera module is the first specification to check, because no amount of host-side processing recovers a channel the sensor already clipped.
Onboard ISP vs. Host ISP: Where the Processing Should Happen
Every design puts an ISP somewhere. The onboard ISP vs host ISP question is only about which side of the interface it sits on.
A camera module with built-in ISP hands you finished frames. It runs black level correction, demosaicing, white balance, noise reduction and HDR internally, then outputs processed video. Vadzo’s USB 3.x lineup is built as a camera module with built-in ISP specifically to reduce host load.
A host-side ISP hands you raw data instead. The camera streams Bayer frames and your SoC’s ISP block, or your own code, does the processing.
The practical difference is integration time. A UVC camera ISP is driver-free and plug-and-play, so you get processed video on day one without writing a pipeline or tuning a sensor. That UVC camera ISP path is why the Falcon and Merlin series integrate as quickly as they do.
A MIPI camera ISP arrangement usually works the other way round. On a MIPI camera ISP design the module streams raw into an SoC such as Jetson, which already has ISP hardware waiting, so the processing moves rather than disappears.
The cost side of onboard ISP vs host ISP is control. Once processing happens on the camera, you work within the feature set that module exposes.
When Raw Bayer Output Is the Right Call
Raw output earns its place when your application is the consumer, not a human viewer.
If you are training or running a custom AI pipeline, a tuned ISP is working against you. Noise reduction and sharpening are tuned to look correct to an eye, and that is not the same as preserving the signal your model was trained on. The YUV vs RAW output camera choice is really a choice about who the image is for.
Vadzo covers this case directly. The Bolt-2020BRS streams 8-bit Bayer raw over 4-lane MIPI CSI-2 with sub-10 ms latency and leaves all demosaicing to the host, and the Falcon-2020BRS streams 5120 x 3840 Bayer data over USB 3.2 Gen 1 with dynamic ROI support.
The AR2020 sensor is offered in both forms, so the same 20MP Bayer pattern sensor is available as a raw camera or as an ISP-processed one depending on which side of this decision you land.
Which ISP Camera for Embedded Vision Fits Your Design
Vadzo’s Falcon and Merlin series are the onboard-ISP lineup. Eleven SKUs across USB 3.2 Gen 1 and USB 2.0 output processed video directly, so each one is an ISP camera for embedded vision work where host compute is already committed to inference.
Camera | Sensor | Resolution | Interface | Onboard ISP | Documented ISP Features |
Onsemi AR1335 (13MP) | 13MP (4K) | USB 3.2 Gen 1 Type-C | Yes | ROI-based AE and AF, Digital PTZ, iHDR | |
Onsemi AR0830 (8.3MP) | 8MP (4K) | USB 3.2 Gen 1 Type-C | Yes | Wake-on-Motion, LI-HDR, eDR | |
Onsemi AR0821 (8.3MP) | 8MP (4K) | USB 3.2 Gen 1 Type-C | Yes | HDR, Auto White Balance, Auto Exposure | |
Onsemi AR0544 (5MP) | 5MP | USB 3.2 Gen 1 Type-C | Yes | Low Power, Wake-on-Motion, eDR, LI-HDR | |
Onsemi AR0521 (5MP) | 5MP | USB 3.2 Gen 1 Type-C | Yes | — | |
Onsemi AR0522 (5MP) | 5MP | USB 3.2 Gen 1 Type-C | Yes | — | |
Sony IMX662 (2MP Starvis 2) | 2MP (1080p) | USB 2.0 Micro B | Yes | Onboard dewarping for 360° M12 lens, HDR | |
Sony IMX291 (2MP Starvis) | 2MP (1080p) | USB 2.0 Micro B | Yes | — | |
Sony IMX290 (2.13MP Starvis) | 2.13MP (1080p) | USB 2.0 Micro B | Yes | — |
A dash means the product page does not itemize ISP features for that module. The camera ISP is present; ask the engineering team for the full feature list before you specify one of these.
Raw output cameras: host does the processing
Camera | Sensor | Resolution | Interface | Onboard ISP | Output |
Onsemi AR2020 (20MP) | 20MP (5120 x 3840) | USB 3.2 Gen 1 Type-C | No | 8-bit Bayer raw, Dynamic ROI, low latency | |
Onsemi AR2020 (20MP) | 20MP | 4-lane MIPI CSI-2 | No | 8-bit Bayer raw, sub-10 ms latency |
Read the split by interface. Seven of the eight USB 3.x cameras carry an onboard camera ISP, all four USB 2.0 modules do, and the single MIPI CSI-2 module is a raw camera.
That mapping is not accidental. USB is where a UVC camera ISP gives you plug-and-play processed output. MIPI is where a MIPI camera ISP design connects straight into an SoC that already has an ISP block.
Choosing an ISP camera for embedded vision therefore starts with your host, not with the sensor. If your SoC has no usable ISP block, a camera module with built-in ISP is the shorter path. If it does, raw output keeps the tuning in your hands.
Frequently Asked Questions
What is an ISP in a camera?
Asking what an ISP in a camera is really means asking what turns sensor measurements into a picture. A Bayer pattern sensor records only one color per pixel, so its output is a mosaic rather than an image. The camera ISP runs black level correction, lens shading correction, the demosaicing algorithm that performs RAW-to-RGB conversion, noise reduction, color correction, and gamma, then outputs YUV or MJPEG. It also runs the 3A algorithms for auto exposure, auto white balance and auto focus. Vadzo’s Falcon and Merlin USB cameras do all of this on the module, so your host receives finished frames.
Do I need an ISP in my camera module?
The question do I need an ISP in my camera module comes down to where you want the processing to run. If you need working video quickly and want host compute free for your application, a camera module with built-in ISP is the direct route, and a UVC camera ISP needs no driver work at all. If you are building a custom AI pipeline and want control over the demosaicing algorithm and color science, raw output serves you better. The trade-off is real: an onboard camera ISP saves integration time but limits you to the features it exposes. Vadzo covers both sides, with eleven onboard-ISP SKUs across the Falcon and Merlin series and two raw Bayer cameras on the AR2020 sensor.
What is the difference between an ISP and an image sensor?
The difference between an ISP and an image sensor is capture versus interpretation. An image sensor converts photons into electrical measurements and outputs them as a Bayer pattern sensor mosaic, where every pixel carries one color value. The camera ISP is the processing block that reconstructs the missing color through RAW-to-RGB conversion and corrects for sensor and lens behavior. Because the difference between ISP and an image sensor is a division of labor rather than a single part, the same sensor can appear in both an ISP camera and a raw camera. Vadzo’s AR2020 lineup shows this directly, with the same 20MP sensor offered as raw output on the Bolt-2020BRS and Falcon-2020BRS.
Can I use a camera without an ISP?
Yes, whether you can use a camera without an ISP is a fair question, and the answer is that your host does the work instead. Cameras like the Bolt-2020BRS and Falcon-2020BRS stream 8-bit Bayer raw and leave all demosaicing and color processing to the host, which is what you want when you need direct sensor data access. Platforms such as Jetson have their own ISP hardware, so on that kind of MIPI camera ISP design, the processing moves rather than disappears. Budget for it, honestly, because a raw pipeline has to be built and tuned before you can evaluate image quality. If your team has the pipeline and wants maximum fidelity, the Bolt-2020BRS delivers raw Bayer over 4-lane MIPI CSI-2 with sub-10 ms latency.
Does an onboard ISP reduce host processing load?
Yes, and that is the main reason to choose one in the onboard ISP vs host ISP decision. When the module handles the demosaicing algorithm, the auto white balance camera loop, the auto exposure control camera loop, the noise reduction image sensor stage and HDR internally, none of that work lands on your SoC. Vadzo’s USB 3.x lineup is designed this way specifically to reduce host load, delivering processed video at the camera output. That freed compute goes to your application, which matters most on an image signal processing embedded camera platform where inference is already the constraint. Across the Falcon and Merlin series, eleven Vadzo SKUs offer this architecture over USB 3.2 Gen 1 and USB 2.0.
