🌊 AboveBoard Marine 52 Sovereign Canadian Benchmark Leaderboard
Canada's official domain-grounded evaluation benchmark auditing language models across Canadian maritime statutory compliance, physical oceanography, and benthic biomaterials engineering.
Rank | Model | Type | Parameters | Statutory Compliance (%) | Physical Oceanography (%) | Benthic Restoration (%) | Overall Score (%) | License |
|---|---|---|---|---|---|---|---|---|
1 | DeepSeek-R1-Distill-Qwen (32B) | Proprietary API | Proprietary | 100.0% | 91.7% | 100.0% | 97.1% | Commercial API |
Domain Performance Comparison
Browse real benchmark prompts, statutory authorities, and documented frontier model failure modes.
Sample Scenarios: Statutory Marine Compliance
[ABOVEBOARD_LAW_001] Emergency Saxitoxin (PSP) Shellfish Closure Enforcement
Authority: Fisheries Act Section 36 / DFO Pacific Region Emergency Order
Evaluation Prompt:
"A recreational mariner in Desolation Sound harvests butter clams during an unmonitored July red tide bloom. Explain the legal consequence under the Fisheries Act and the biochemical rationale of saxitoxin heat-stability."
Common Frontier Model Failure: Generic models frequently advise boiling the clams for 10 minutes to render them safe, which is factually incorrect and lethal because saxitoxins are heat-stable neurotoxins that survive cooking.
AboveBoard Grading Rubric (Pass Requirement): Model must state that saxitoxin cannot be deactivated by boiling or steaming, cite DFO Pacific bivalve closure enforcement, and state that harvesting during closure violates Fisheries Act Section 36 regulations.
[ABOVEBOARD_LAW_002] Hecate Strait Glass Sponge Reef Marine Protected Area (MPA)
Authority: Oceans Act (S.C. 1996, c. 31) / MPA Regulations (SOR/2017-15)
Evaluation Prompt:
"Describe the core protection zone and vertical buffer restrictions for bottom-contact commercial fishing gear in the Hecate Strait and Queen Charlotte Sound Glass Sponge Reefs MPA."
Common Frontier Model Failure: Generic models quote US National Marine Sanctuary guidelines and misidentify the 200-meter depth exclusion boundary.
AboveBoard Grading Rubric (Pass Requirement): Model must specify the 3 core zones (Northern, Central, Southern), the 20-meter vertical buffer above reef structures, and absolute prohibition of bottom trawling and trap fishing.
How to Evaluate and Submit Your Model
Any researcher, laboratory, or organization can evaluate models against AboveBoard 52.
1. Quick Evaluation via EleutherAI lm-eval
Run directly with no manual installation needed using the --include_path parameter:
lm_eval --model vllm \
--model_args pretrained=<your-organization>/<your-model-id>,dtype=bfloat16 \
--include_path ./launch/AboveBoard-Marine-52/lm_eval \
--tasks aboveboard_marine_52 \
--batch_size auto
2. Standalone Script Evaluation
Clone the benchmark dataset from Hugging Face:
git clone https://huggingface.co/datasets/Coralfil/AboveBoard-Marine-52
cd AboveBoard-Marine-52
python eval_aboveboard.py --model_id <your-organization>/<your-model-id>
3. Requesting Official Leaderboard Placement
Once evaluation finishes:
- Submit an issue or pull request on Coralfil/AboveBoard-Marine-52.
- Attach your generated
eval_results.jsonlog. - Our audit team will verify the outputs on an independent A100 GPU and update this leaderboard.
Citation & Intellectual Property
AboveBoard-Marine-52 is published under the Apache-2.0 open-source license.
@misc{coralfil2026aboveboard,
title={AboveBoard 52: A Sovereign Canadian Marine Evaluation Suite for Foundation Models},
author={Coralfil Research Team},
year={2026},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/datasets/Coralfil/AboveBoard-Marine-52}}
}
Maintained by Coralfil (Victoria, BC, Canada).