Upcoming events

Reproducibility initiative at ASPLOS'20
Website: Link
Date: March 16, 2020
Deadline: December 4, 2019
Reproducibility initiative at the Conference on Systems and Machine Learning 2020
Website: Link
Date: March 2, 2020
Deadline: January 10, 2020
Reproducibility initiative at SuperComputing'19
Website: Link
Date: November 17, 2019
Student Cluster Competition at SuperComputing'19
Website: Link
Date: November 17, 2019
Dashboard (prototype): Link
Airbus Quantum Computing Challenge'19
Website: Link
Date: October 31, 2019
MLPerf v0.5 inference benchmark submissions
Website: Link
Date: September 6, 2019
Notes: MLPerf.org announced the v0.5 inference benchmark suite. Submissions for two image classification tasks, two object detection tasks, and one machine translation task are due September 6th, 2019! Other resources: InsideHPC
Validation of MLPerf inference benchmark results
Website: Link
Date: September, 2019
Workflows:
Dashboard (CK unification and reproducibility): Link
Notes: MLPerf.org announced the v0.5 inference benchmark suite. Submissions for two image classification tasks, two object detection tasks, and one machine translation task are due September 6th, 2019! Other resources: InsideHPC, GitHub.

Recently reproduced papers with unified workflows

AGGREGATHOR: Byzantine Machine Learning via Robust Gradient Aggregation
Georgios Damaskinos, El Mahdi El Mhamdi, Rachid Guerraoui, Arsany Guirguis, Sebastien Rouault
Optimizing DNN Computation with Relaxed Graph Substitutions
Zhihao Jia, James Thomas, Todd Warszawski, Mingyu Gao, Matei Zaharia, Alex Aiken
Highly Efficient 8-bit Low Precision Inference of Convolutional Neural Networks with IntelCaffe
Jiong Gong, Haihao Shen, Guoming Zhang, Xiaoli Liu, Shane Li, Ge Jin, Niharika Maheshwari, Evarist Fomenko, Eden Segal

Recently added standardized workflows

repo:armnn-mlperf
Description:
CK workflows to unify and automate ArmNN preparation and benchmarking for mlperf.org
Workflow:
programs
Tasks (program workflows):
repo:ck-mlperf
Description:
Automated, customizable and portable CK workflows and components for the MLPerf benchmark.
Workflow:
programs
Tasks (program workflows):
repo:ck-scc18
Description:
CK workflow and components to automate SuperComputing'18 Student Cluster Competition.
Workflow:
reproducible paper - passed Artifact Evaluation:

Tasks (program workflows):