By Maryna Poplavska, Head of Project Managers, Trembit — real-time video, telemedicine and moderation delivery
In image moderation, a human review band is the range of classifier scores between an auto-reject line and an auto-allow line, where an item is held for a person. On a 1:1 video platform we built, we shipped none. The photos users uploaded got one verdict against one threshold: at or above it the photo was rejected, below it it published immediately, and nothing was ever held for a moderator. A moderator role and an admin reports queue existed. But no person decided a borderline photo before publication. The delivery team described it as deliberate. We have no outcome data, so this is a design and its costs, not a claim it was right.
Key takeaways
- A band is a cost, not a default: a third state needing a queue, people, a time limit, an expiry rule and a message to the uploader.
- This build was binary, synchronous and fail-closed: one number decided everything, and a vendor error left the photo
pending.- With no band, the threshold is the policy. This build set it at 0.85 on a commercial classifier — a number only its own model can interpret.
- No band meant no person decided a borderline photo before publication. The build had a moderator role and an admin reports queue — but nothing stood in for the band, and reporting after the fact catches only what got through: no other user sees a rejected photo.
- We do not know which photos it wrongly rejected, or whether 0.85 was the first value. Nothing here says the design worked.
Engineers: the threshold and fail-closed sections. Product and trust-and-safety leads: the cost, where the human goes, and when skipping it is wrong.
What did we build instead of a human review band?
One gate, one number, two outcomes. The photos users uploaded went to a commercial image classifier. The delivery team’s description, code-verified in October 2026: there is no borderline band and no human queue for photos at all. The verdict is binary: at or above the threshold the photo is rejected, below it it publishes immediately, and nothing is ever held for a moderator.
Three further facts:
- Threshold 0.85, carried as an environment knob rather than a constant.
- Synchronous. The upload waits for the classifier’s verdict — no accept-now-check-later window.
- Fail-closed: if the vendor errors, the photo stays
pendingindefinitely rather than going live unmoderated.
So the system does hold photos in a third state, pending — but the cause the source names is the vendor failing, not the classifier being unsure.
Does image moderation need a human review band at all?
Not as a technical requirement — vendors sell both shapes. Sightengine’s FAQ offers a fully automated workflow in which “no human moderators will be involved in the process”, alongside a hybrid one routed by thresholds; Stream’s docs describe a default that accepts the image first, checks it in the background, and puts flagged images in a review queue.
Our own published position favours a band, and this piece does not retract it. Our live-video moderation pipeline write-up describes “a middle band (roughly 0.70–0.95) that routes to human review” as a common setup. For borderline live-video scores we still favour a band — ours, not that page’s finding: a wrong automated call mid-session ends something you cannot un-end.
A photo at an upload gate is different: nothing is in flight, so holding it damages no session. In our reading that makes a band cheaper to hold, not unnecessary: it buys somewhere to put uncertainty, and the next section prices it.
What does a human review band cost to run?
Our analysis, not the team’s reasoning. A band is not a threshold change; it is a new state in the product, and every state has to be specified and staffed:
- A queue, with its own admin surface, permissions and audit trail.
- Someone to work it inside a time limit — a staffing number, not a config value.
- A rule for when the limit passes. Auto-publishing after N hours is fail-open by another name; holding it makes the uploader wait with no end date. Either way it is policy.
- A message to the uploader meanwhile, with its support cost.
The build above has a held state, pending, but what the source puts there is a vendor error, not an uncertain score. None of the four exists for a review state. That is what skipping a band saves — an operational surface, not inference cost. We also built a moderation system with a human review queue for borderline content on a different platform, whose write-up notes that “Upload moderation is a queue with a flexible latency budget.” Same company, opposite design.
How do you set a moderation threshold with no review band?
With no band the middle has zero width, so one number carries the whole policy. This build set it at 0.85, a configuration value rather than a constant in code.
What does a published starting point tell you? Two vendors publish one, on scales that differ. Hive’s visual-moderation docs say “Generally, a threshold model confidence score of 0.90 is a good starting point to flag sensitive content in a positive class, but you can adjust based on how the model performs on a natural distribution of your data.” Stream’s thresholds run 1 to 100, with a similar instruction: “Start with the default (50) and adjust from what you see in the review queue.” A confidence score belongs to the model and class that emits it, so one deployment’s starting point is not another’s bar. What transfers is the instruction both of them give: adjust against how the model performs on your data — and that record is what we do not have.
Where an error lands follows from the line (ours): on any one classifier a lower line rejects more and a higher one publishes more. A band absorbs part of that choice; without one, the number is it.
Set it beside our own published post and the number gets uncomfortable: the common live-video band it describes runs roughly 0.70 to 0.95, so 0.85 sits inside it — loosely, since scores differ by classifier. On live frames that post routes that range to a person; here anything at or above the line was rejected outright. The same post calls cutoffs “a policy choice, not a technical constant”, and this is a different medium — but the two numbers belong in one sentence.
Should photo moderation fail open or fail closed when the API errors?
This build fails closed. The upload waits for the verdict, and on a vendor error the photo stays pending indefinitely rather than going live unmoderated. In our reading that is the stronger default, and it echoes OWASP’s secure product design guidance: “Design systems to fail in a secure state, rather than exposing vulnerabilities when they malfunction.” Applying a security principle to a moderation gate is our analogy.
The cost: your upload path inherits the vendor’s availability, and an outage becomes a stuck state, not a silent publish. The delivery team’s own description says “indefinitely” and is silent on whether a timeout, retry, alert or sweep exists, so we assert nothing about them — but a stuck state with no owner is the failure mode to design for.
Where does the human go when there is no review band?
Nowhere that replaces it, though the build was not without a moderation route. It carried a MODERATOR role and an
admin reports queue. The handover’s list of faults fixed in the release it delivered includes
“Moderation reports arriving late”, cause “No polling on the admin queue” — so the queue was
there and somebody noticed it was not refreshing. In our reading that means it was being used; we do
not know by whom, how often, or on real reports.
None of which stands in for a band. A report arrives after publication by definition, and whether any after-the-fact route can reach a photo is one of the unestablished things below.
Three things are not established and this piece does not assert them: whether a report can target a photo as against a user or a chat, whether a rejected uploader can contest the verdict, and whether any appeal path exists.
A band is pre-publication control: a human sees the item before anyone else can. A reports queue is post-publication: someone else saw it first and objected. The asymmetry is the point. A band puts a person in front of the uncertain photo first, so either kind of error can be caught there. A queue of reports catches only what was wrongly let through, because no other user sees a rejected photo. In our reading, nothing in between handles “the model was unsure”.
When is skipping the band the wrong call?
Three tests, all ours. Fail one and build the band.
- A wrong rejection is expensive or hard to recover — the uploader cannot retry cheaply, or the item is time-sensitive.
- Legitimate content routinely resembles violating content — our case-study platform hit this with medical and artistic imagery.
- A duty to review, report or explain attaches to the category. A classifier gate makes a decision; it may not discharge a duty. Scope that with counsel.
Run them on this build and all three come back blank. Test 1 needs what a wrong rejection cost an uploader, test 2 the false-positive record, test 3 the client’s legal position. We hold none, so this is a design we can describe — not one we can say passed its own tests.
What don’t we know about how this design performed?
Two things, recorded as unknowns rather than smoothed over.
We do not have the field record of which photos it wrongly rejected, and the delivery team’s reply did not carry it. That record would tell you whether 0.85 was defensible here. In our reading, the gap comes from the design. A band produces the number as a by-product: every held item a person overturns is a labelled error. With no band and no human queue for photos, that source does not exist, so the rate would have to come from complaints, tickets or a deliberate sample.
We also do not know whether 0.85 was the first value or a tuned one — nobody has told us which. Tuned, the number is evidence about this content type. Set once and left, it is evidence about nothing.
FAQ
Can image moderation run with no human review band? Technically yes — we shipped one photo-upload build that way and cannot say whether it was right. For borderline live-video scores we favour a band. It depends on how expensive a wrong rejection is, how often legitimate content resembles violating, and whether a duty to review attaches.
What threshold should image moderation use? Vendors publish starting points on their own scales — Hive 0.90; Stream a default of 50 out of 100 — and both say to adjust against your own data. That is guidance about method; the number does not carry.
Should a moderation API fail open or fail closed? Fail-closed means nothing publishes unchecked when the vendor errors; fail-open means it does. This build fails closed, which costs availability: an outage becomes a stuck state, not a silent publish.
Does “no human review band” mean no human moderators? It means no person decided a borderline photo before publication. The build had a moderator role and an admin reports queue — we do not know who used them, or how often. What the delivery team described is that nothing was ever held for a moderator at the upload gate. A report can only flag something published, and no other user sees a rejected photo.
The nearest comparison is above: the platform where we did build the queue. Neither choice is called right.
Running both an upload path and live calls? Bring both paths and the thresholds you run today to a free 30-minute call, and we will work out where the human belongs in each — no deck, no pitch. Talk to our team, or start with AI inside live video and voice.
