In this talk, Grigori Fursin describes his 10-year effort to solve numerous reproducibility issues in ML and systems research and make it easier to use it in the real world. He will share his experience reproducing 150+ research papers during artifact evaluation at multiple ACM conferences. This tedious experience motivated him to develop the Collective Knowledge framework and the cKnowledge.io portal to bring DevOps principles to CS research. He will also present portable CK workflows with plug & play components to package and share research artifacts and results with common Python APIs, reusable automation actions and unified meta descriptions. Such workflows can be used to automatically build, benchmark and validate research techniques across continuously evolving technology stacks.
Fursin will conclude with several practical use-cases of this technology to automate benchmarking, optimization, and co-design of efficient computer systems in collaboration with Arm, IBM, General Motors, ACM, the Raspberry Pi foundation, and MLPerf. His long-term goal is to help researchers share their new research techniques as portable and production-ready packages along with published papers that can be quickly validated by volunteers and adopted by industry.
Grigori Fursin, President, cTuning Foundation; Founder, cKnowledge.io; ACM Taskforce on Reproducibility
