MOSAIC Community Challenge: Strains

Benchmark and improve computational methods in microbiome characterization at the strain level

We are excited to announce the winners of the first Mosaic Community Challenge, Strains #1:

Learn how these researchers participated in the challenge by watching the following webinar:

Mosaic Community Challenge: Strains #1 is now closed. We're reviewing the results and will make them available to the submitters for review. Results will be made available to all participants in 3-4 weeks.

Accelerate development of computational methods that can unravel the microbiome at finer resolution

You should participate if you are a:
  • Method Developer
    • benchmark your method without spending your time installing others’ methods
    • focus your time improving your method
    • share your method with the entire community (only if you want to!)
  • Microbiome Scientist in Academia
    • find and test methods most suitable for your dataset
    • access to community-validated methods
    • guarantee reproducibility
  • Microbiome Scientist in Industry
    • test and integrate community validated methods (appropriate license applicable) into your own pipelines
    • enable frictionless portability into secure and GXP-compliant environment for clinical research
    • collaborate by sharing data and pipelines between teams
  1. Create Mosaic account and receive $50 worth of compute credits if you do not have a DNAnexus account yet!
  2. Read through the Rules of Participation and accept the Challenge.
  3. Work on the Challenge by either:
    • Creating your app on Mosaic, run it FOR FREE and submit results in the required format (and if you choose to share your method), or,
    • Downloading the datasets, working outside Mosaic and submitting the results in the required format
  4. Learn more about submitting your results.

Methods will be evaluated with the following metrics:

  • Profiling: Precision, Recall and F1-score
  • Assembly: Genome fraction, misassemblies and indels+mismatches (per 100kb)
  • Binning: Precision, Recall and Adjusted Rand Index

These metrics are by no means exhaustive, rather are meant to create community discussion around appropriate metrics for evaluating different types of methods.

See Evaluation metrics for details.

  • Open-beta: November 13, 2017, 12:00 p.m. ET
  • Challenge launch: December 1, 2017, 12:00 p.m. ET
  • Challenge end: March 21, 2018, 11:59 p.m. ET March 30, 2018, 11:59 p.m. ET
  • Winners announced: April 2018

The microbiome industry is gearing up to introduce microbial-based products. Profiling a microbiome sample is the fundamental step in any microbiome analysis, but becomes even more challenging when higher resolution really matters. Current state-of-the-art methods based on next-generation sequencing are able to predict organisms down to the genera and species level. However, within the same species, individual strains can elicit different functionality and metabolic response, making strain-level identification and quantification key to the development and tracking of microbiome-based health solutions. It is critical to be able to accurately determine the type and quantity of microbes in a sample at the strain level in order to bring safe and effective products to market, and to accurately monitor their status within the human body. Participants will contribute to the improvement of sequencing standards while also testing their own tools and methods in a secure and collaborative environment.

Previous Webinars
  • Webinar 2
  • Title: Building Apps on Mosaic
  • Date: November 30, 2017

  • Webinar 1 - Coming soon
  • Title: Introduction to Mosaic Community Challenge: Strains #1

Janssen Research & Development, LLC, through its Janssen Human Microbiome Institute, provides the vision, strategy, and design of Mosaic community challenges to seek answers to specific questions that are intended to speed the translation of science into novel products.

Mosaic has been developed and is maintained by DNAnexus Inc., a leader in cloud bioinformatics. DNAnexus has extensive experience in building secure and scalable platforms that enable computational analyses of biological data on the cloud.