Detecting packages with RF-DETR

Finding packages in an image is a basic building block for logistics and warehouse automation: counting parcels on a shelf, checking a delivery photo, or spotting a box left on a doorstep. In this project I took an off-the-shelf detector, fine-tuned it on a public dataset, measured how it compares with an existing model, and wrapped it in a small web app.

The code, notebook and results are on GitHub: carlos-dev-research/package-detect.

Overview#

flowchart LR
  A[Packages V4<br/>Roboflow Universe] --> B[Fine-tune<br/>RF-DETR Base]
  B --> C[Evaluate<br/>mAP, precision, recall]
  C --> D[Gradio app]

1. Data: a public, single-class package dataset in COCO format.
2. Model: RF-DETR Base, pretrained on COCO.
3. Fine-tuning: 10 epochs on the package dataset.
4. Evaluation: mAP, precision and recall on the held-out test split, at two confidence thresholds.
5. Deployment: a Gradio interface for trying the model on any image.

Data#

I used Packages V4 (version 7) from Roboflow Universe. It has a single class, package, and comes with train, validation and test splits. I downloaded it in COCO JSON format, which RF-DETR's trainer reads directly:

rf = Roboflow(api_key=os.getenv("ROBOFLOW_KEY"))
project = rf.workspace("ammar-s").project("packages-v4")
dataset = project.version(7).download("coco")

Before training, I plotted a few images with their ground-truth boxes using pycocotools to check that the annotations lined up with the boxes.

Why RF-DETR#

RF-DETR is a DETR-style detector from Roboflow. Instead of predicting offsets from a grid of predefined anchor boxes like classic YOLO-style detectors, it uses attention to predict a set of objects directly. That means there are no anchor sizes to tune for a new dataset. It is also designed for real-time inference, and Roboflow's benchmarks show it performing well on COCO. The Base variant has about 32 million parameters, small enough to fine-tune quickly on a single GPU.

Fine-tuning#

Starting from the COCO-pretrained checkpoint, training is a single call:

from rfdetr import RFDETRBase

model = RFDETRBase()
model.train(
    dataset_dir=dataset.location,
    epochs=10,
    batch_size=16,
    grad_accum_steps=1,
    lr=1e-4,
    early_stopping=True,
)

The trainer evaluates on the validation split after every epoch and keeps the best checkpoint. It tracks both the raw weights and an exponential moving average (EMA) of them, which is usually a little more stable.

The curves show most of the gain in the first two epochs, as expected when starting from a strong pretrained model. AP@50 rises from about 0.93 to 0.96–0.97 and stays there. Validation loss is noisy, with a spike at epoch 7, but the detection metrics recover by the final epochs. With a dataset this size, a few more epochs or a lower learning rate would likely smooth this out.

Evaluation#

The metrics#

For a detection to count as correct, its box has to overlap a ground-truth box enough, measured as intersection over union (IoU). With true positives ($TP$), false positives ($FP$) and false negatives ($FN$):

Precision answers "when the model says package, how often is it right?", and recall answers "of all the packages, how many did it find?". mAP@50 is the area under the precision–recall curve with a match threshold of IoU $\ge 0.5$. mAP@50:95 averages that over stricter IoU thresholds from 0.5 to 0.95, so it also rewards tight boxes.

Results on the test split#

At the end of training, the best checkpoint scored:

Choosing a confidence threshold#

The detector outputs a confidence score for every box, and the application decides which ones to keep. I ran the test split (46 images) through the model at two confidence thresholds, applied non-maximum suppression, and computed the metrics with supervision. For reference, I compared against the model published on the dataset's Roboflow Universe page, using the numbers reported there:

• At 0.25, RF-DETR finds almost every package (98.5% recall) and edges out the reference model on mAP@50, at the cost of more false positives.
• At 0.50, it becomes slightly more precise than the reference model (97.0%) while still beating it on recall (94.1% vs 91.1%).

The threshold is a product decision, not just a model setting. A system that must not miss a parcel, such as counting deliveries, wants the low threshold. A system where false alarms are expensive, such as triggering a robot arm, wants the high one. With a single model you can serve both.

Deployment#

To make the model easy to try, deploy.py loads the best checkpoint, optimizes it for inference, and wraps it in a Gradio interface. You upload an image and get it back with a box and confidence score for each detected package:

model = RFDETRBase(pretrain_weights="./output/checkpoint_best_total.pth")
model.optimize_for_inference(compile=False)

def predict(image):
    detections = model.predict(image)
    return plot_detections(image, detections)

gr.Interface(
    fn=predict,
    inputs=gr.Image(type="pil", height=500),
    outputs=gr.Image(type="pil", height=500),
    title="RF-DETR Package Detection",
).launch()

Even with boxes stacked and partially overlapping, each package gets its own detection at 0.93–0.94 confidence.

Takeaways#

• Pretrained transformers transfer well. Ten epochs on a small single-class dataset were enough to match or beat a dedicated reference model.
• Report metrics at the threshold you'll deploy. The same model is either the most precise or the most thorough option, depending on one number.
• Next steps: a larger and more varied dataset (warehouse shelves, conveyor belts, poor lighting), longer training with a learning-rate schedule, and measuring latency on edge hardware.

Resources#

• Code and notebook: github.com/carlos-dev-research/package-detect (Apache 2.0)
• Dataset: Packages V4 on Roboflow Universe
• Model: RF-DETR and its training guide