Jev Image Analysis: 5 Test Results That Matter

Jev image analysis answers one narrow question well: can a fast decision model look at a picture and pick the right option without human training? The speaker's tests say mostly yes on speed and high-confidence accuracy, with a documented catch around the box question type and calibration at low confidence.

What Jev image analysis does and who can use it

Jev image analysis is a beta feature, announced in October 2026, that lets the Jev decision platform accept images as input and answer structured questions about what they show, per the video from The AI Automators. Before this release, Jev could only decide based on text. A second addition, a question type called "box", returns the coordinates of an object in the image.

The feature is available to all users of the Jev platform in beta, and the vendor has said it will keep improving it over the coming weeks. Those are platform claims from the release announcement cited in the video; this article cannot independently verify the product's current status beyond the video's date.

The practical difference from classic computer vision is the setup cost. Traditionally you would train a specialized vision model for each task. With Jev image analysis, you describe what to check in plain language, attach an image and get a decision back, with no training step. The trade-off, covered later in this article, is that you depend on the model's calibration rather than on a model tuned to your data.

How the box question type works, and its one big gotcha

The box question type returns four coordinates marking where an object sits in the image, alongside the normal choice and boolean question types. In one playground example from the video, a shelf photo was asked "which item matches this flammable pictogram?" and the model boxed a bottle of high-proof grain alcohol rather than the bleach, correctly reasoning that bleach carries a hazard icon but is not flammable.

There is one gotcha the video stresses: the box type always outputs a box. There is no "not found" answer. If the target is completely absent, the model still returns four edges somewhere in the image.

The recommended workaround follows three steps:

1. Ask a boolean (null) question first: is the target present in the image at all?
2. Check the returned confidence score before trusting the answer.
3. If presence is not confirmed, ignore whatever coordinates the box question returned.

The same shelf demo showed why confidence matters. Asked whether anything on the shelf was drinkable, the model returned 94% true, but because no box question was used it could not pinpoint the item. In a separate damage-assessment example, it rated a cracked phone as "severe" rather than "extreme" with 91% confidence and a probability of 93%, matching the criteria definitions given in the prompt.

First test: 200 public bug reports at 2 cents per batch

The video creator built a support-ticket app that checks whether an attached screenshot actually backs up the reported problem. For the public dataset, he used 200 tickets drawn from SWE-bench-style public bug reports, a benchmark collection of real software issues documented at swebench.com.

All speaker-reported results, from The AI Automators video: the 200-image batch classified in just under 4 seconds at a cost of about 2 US cents, with bounding boxes drawn on each flagged screenshot. Of the 200 cases, 79 were accepted, 112 were routed to human review, and the remaining 9 are not accounted for in the narration, so treat the two published counts as partial rather than a complete partition.

His conclusion is a scoping one, and it is worth quoting the logic: these decision models are not calibrated enough to run this workflow fully hands-off. He frames them as a very good first pass, a cheap triage layer, with humans handling the uncertain middle.

Second test: 1,000 synthetic cases, accuracy and calibration

To get a ground truth, the creator generated 1,000 controlled tickets with synthetic screenshots and known correct answers. Running them live took about 20 seconds at roughly 50 tickets per second, in batches of 16, for a total cost of 11 US cents. A confidence cutoff of 65% decided what could be automated.

The outcomes split three ways, per the speaker's own count:

Outcome	Cases	Handling
Wrong screenshot attached	724	Routed back to customer
Correct screenshot	175	Accepted automatically
Below 65% confidence	101	Sent to human review

Accuracy

With ground truth known, the model answered 934 of 1,000 cases correctly, a raw accuracy of 93.4%, with 66 wrong. That figure belongs to this specific synthetic dataset and prompt design, not to the platform in general.

The confidence gate

Of the 899 automatic decisions above the 65% threshold, the model made zero mistakes. All 66 errors fell inside the 101 cases sent to human review. For this workload, the confidence score separated reliable answers from unreliable ones cleanly.

Calibration

Calibration asks whether the stated probabilities match reality. On the 899 automatic decisions, the average stated probability was 95.3%, implying roughly 42 expected mistakes; the actual count was zero, so the model was underconfident there. On the 101 reviewed cases, the average probability was about 48% while actual accuracy was around 34%, meaning 66 mistakes against roughly 52 expected, so it was overconfident exactly where humans were needed. These are speaker-reported numbers from a single synthetic run, not an independent evaluation.

Five patterns for putting images inside a request

Because images can sit anywhere in the request, in the state, the instructions or the criteria, the video demonstrates several reusable patterns. All of them come from the creator's demonstrations rather than vendor documentation:

• Compare against a known picture: ask whether the image in the state shows the same person, product or place as a reference image you supply in the criteria.
• Define answer options as images: in one demo, the criteria were nothing but meme pictures, and the model picked the appropriate reaction to a hypothetical database fire.
• Detect what changed: send the previous frame of security footage and ask what differs, such as a new person or animal appearing.
• Tile high-resolution images: cut a large image into tiles, ask a choice question over the tiles, then send a higher-resolution crop of the chosen tile for detailed questions.
• Filter and route: the support-ticket app itself, which sends low-confidence or unsupported screenshots back to the submitter.

One boundary the video repeats: this is not a generative text model. You must provide options for it to choose between, which is what keeps it fast and cheap but limits it to classification-style problems.

Limits, pricing reality and alternative decision models

The stated limits, as read out in the video: still images only, PNG or JPEG, a 16 MB maximum per image, a 32 MB maximum request body, and minimum and maximum resolution constraints. Enabling the feature requires setting a header flag, described in the video as "type safe danger zone images" set to true. Most size limits can be worked around by resizing or cropping images before the request.

On cost, the only figures available are the speaker's own runs: about 2 US cents for 200 images and 11 US cents for 1,000. Those are single-run observations on one account and prompt design, not published pricing.

Jev is not the only option in this space. The video names Cloudflare as offering a decision model of its own, and mentions open-source decision models built on a Qwen base and a Gemma base that also accept image input. The transcript's specific model names for those open models could not be verified against a primary source, so they are omitted here; the creator's earlier video compares them directly against Jev. Anyone evaluating this category should run that comparison on their own data, since every number in this article traces back to one person's tests.

FAQ

• Is Jev image analysis available to everyone? According to the October 2026 release covered in the video, the feature is in beta and available to all users of the Jev platform. Verify current availability on the platform's own documentation before planning a deployment.

• How accurate was Jev image analysis in the video's tests? On the creator's 1,000 synthetic cases with known answers, it scored 93.4% raw accuracy, and it made zero mistakes on the 899 answers above its 65% confidence cutoff. This is one speaker-reported run, not an independent benchmark.

• Why did bounding boxes appear on objects that were not in the image? The box question type always returns coordinates, even when the target is absent. The recommended pattern is to confirm presence with a boolean question first and discard the box output when presence is not confirmed.

• What image formats and sizes does the feature accept? Per the video, PNG or JPEG only, up to 16 MB per image with a 32 MB request-body limit and resolution bounds. Pre-processing images, such as cropping or resizing, addresses most of these limits.

• Can it replace a trained computer vision model? In the tested workflow it worked well as a cheap first-pass triage layer, with 101 of 1,000 cases still needing human review. Whether it replaces a trained model depends on your tolerance for that review gate and on how well its calibration transfers to your data.

Turn your own screen-recorded walkthroughs into articles

This article exists because a practitioner recorded an 11-minute test of a new beta feature, ran 1,200 cases on camera and narrated every number. That kind of hands-on evidence usually stays trapped inside a video, searchable by nobody. If you have the same kind of knowledge in your own uploads, benchmark runs, tool walkthroughs, build logs, Skala Blog turns a YouTube URL into a structured, sourced written article your readers can scan and search.

Source video