ANNOTATION CASE STUDY
Annotation case study
A small image batch.A shared rulebook.
From class definitions to object boundaries, a road-image pilot shows how to align rules, record disagreements and retain the reasons behind each revision.
All case studies
01 / PILOT BRIEF
Align the interpretation before scaling.
A vision team wants to clarify classes, occlusion and truncation before expanding production. Images provide the unit for linking rules, issues and recheck decisions.
- Client profile
- road-vision team
- Task
- 2D bounding-box annotation
- Class scope
- Vehicle, pedestrian, cyclist, traffic sign
- Pilot scope
- 360 images in one pilot batch
02 / EDGE CASES
Three challenges. Explicit review criteria.
Challenge tags identify points to review in 60 sample images. Small objects, occlusion and truncation can overlap in the same image; their counts must not be added as a total image count.
SMALL OBJECTS
Small objects
Distant signs and road users have limited visible detail. Define recognition criteria and exclusions before reviewing classes and box boundaries.
- Images with this tag
- 24
- Total sample images
- 60
Review focus
- Evidence for a class and exclusion rules
- Box boundaries against visible pixels
- Omissions and uncertain small objects
Example recordANN-008 · A traffic-sign boundary was disputed, revised under the candidate rules and passed on recheck.
OCCLUSION
Occlusion
An occluder can hide part of an object. Specify whether boxes follow visible or estimated full extent, supported by attributes and example notes.
- Images with this tag
- 18
- Total sample images
- 60
Review focus
- Visible-box versus full-box task rules
- Occlusion attributes supported by evidence
- Keep uncertain cases unresolved
Example recordANN-037 · The class of a partly occluded road user remains ambiguous and is kept unresolved.
TRUNCATION
Truncation
For objects extending beyond the image, box clipping and truncation attributes need a consistent rule. Truncation and occlusion remain separate concepts.
- Images with this tag
- 12
- Total sample images
- 60
Review focus
- Box clipping at image boundaries
- Separate truncation and occlusion attributes
- No guessed classes for unseen details
Example recordANN-052 · Correct the truncation attribute of a vehicle at the image edge and retain the revision and recheck record.
24 / 18 / 12 are overlapping tag counts, not exclusive scene groups, bounding-box counts or issue counts. Some sample images may have none of these tags.
03 / WORKING METHOD
Turn disagreements into reviewable rules.
Start with a class dictionary and edge-case notes. Keep rule clarification, annotation changes and review decisions distinct. Open items remain in the candidate rules until agreement allows a version to be frozen.
- 01
The same object receives different class interpretations
ApproachDefine class boundaries and exclusions from visible evidence. List ambiguous examples separately instead of forcing a class.
Record retainedClass dictionary, edge-case notes and open issue IDs
- 02
Small, occluded and truncated objects receive inconsistent boundaries
ApproachAgree on box extent and attributes separately. Calibrate with examples rather than combining different challenges into one vague rule.
Record retainedCandidate rule version, image IDs and revision reasons
- 03
Reworked images can be confused with passed images
ApproachTrack initial review, rework, recheck and current status by image. Multiple issues do not cause an image to be counted twice.
Record retainedImage-level review log and revision references
04 / PILOT REVIEW
Record revisions. Keep uncertainty visible.
Review 60 representative images. An image with any unresolved check is excluded from passed images. The remaining 300 images follow the next review schedule.
| Sample range | Reviewed | Initial pass | Recheck pass / rework | Current pass |
|---|---|---|---|---|
| ANN-001–020 | 20 | 18 | 2 / 2 | 20 |
| ANN-021–040 | 20 | 17 | 2 / 3 | 19 |
| ANN-041–060 | 20 | 19 | 1 / 1 | 20 |
| Total | 60 | 54 | 5 / 6 | 59 |
The three sample-ID ranges are disjoint and total 60 images. Rates describe this review only, not box accuracy, batch acceptance or model performance. They are not extrapolated to the other 300 images.
Explore delivery & quality05 / HANDOVER STRUCTURE
Labels need the reasoning that supports them.
Connect image indexes with labels, class rules and review decisions so every revision retains its reasoning.
- 01
Image index & batch scope
Image IDs, file references and batch membership, with image and bounding-box counts kept separate.
- 02
Bounding boxes & attributes
Agreed 2D boxes, classes and occlusion / truncation attributes. These labels are not included in this download.
- 03
Classes & rule versions
Definitions for four classes, boundary examples, exclusions, candidate rules and changes.
- 04
Review & open issues
Image-level sampling denominators, rework and recheck decisions, and unresolved items.
PROJECT TAKEAWAYS
Give the next batch a clear reference.
- 01
Make the rule specific
Class names need boundary examples, exclusions and a defined response when evidence is insufficient.
- 02
Use a consistent counting unit
Count scope and review by image; record box counts separately to avoid mixed denominators.
- 03
Retain the ambiguity
Review open issues with candidate rules, then agree on an executable, frozen version.
LET’S DEFINE YOUR ANNOTATION TASK
Start the pilot with samples and rules.
Describe your image type, draft classes and current challenges to frame a pilot and its review approach.