HackOS 1

Benchmarking Hackathon 1

We ran our first ever benchmarking hackathon on October 25-26 simultaneously at McGill University and at the University of Waterloo. Here are some highlights!

Tracks

We ran four diffferent tracks for this event:

1. Bicycle detection challenge based on a subset of Microsoft COCO
2. Open Chromation Region (OCR) classification as part of Genomic Benchmarks
3. TinyMMLU
4. Whatever floats your boat!

Projects

• Fine-tuning a vision transformer (ViT) to detect bicycles in an MS COCO validation subsample, using synthetic bicycle generation to augment the training dataset. The final validation benchmark got ~85% acc!
• Interpretable ML models for open chromatin region (OCR) detection
• Another open chromatin region (OCR) detection project: Artemis, which used a ConvNet approach
• Unsupervised segmentation for self-driving cars
• WindDM: diffusion modelling for wind maps
• Un poco Coco: training vision transformers (ViT) on augmented data for bicycle detection on a subsample of MS COCO
• Some more work on self-driving cars focusing on semantic scene understanding
• And a demo for benchmarking in the healthcare space!

Shared Takeaways

• Vision transformers with fine-tuning downstream can yield strong model performance
• Revisiting traditional synthetic image geneation methods can be used to increase data availability for downstream model fine tuning
• There is still an anecdotal gap between interpretable, classical methods and high-performing, neural network methods (that tend to lack interpretability)

Hackathon Sponsors & Partners

A special thanks to Rootly (YC S21) for sponsoring, and to Building 21 for the space at McGill!

Topics to Come in Future Updates

• Story of how we organized the first HackOS