Capturing the extent and sources of variability in NGS-based microbiome profiling protocols and analyses

Several studies have documented the significant impact of the multitude of methodological variables in microbiome research on the variability and consequently on the interpretation of NGS-based microbiome data. These differences impede reproducibility of results and the ability of researchers to perform comparisons between independent studies. Understanding the extent of the variability in taxonomic readouts due to differences in protocols is a critical step in development of standardized protocols, reference standards and meta-analyses.

To tackle this issue, the Standards Challenge will provide and ship 700 sets of fecal aliquots and DNA reference material – at no cost to Challenge participants – to any lab in the world.

In return, we ask the Challenge participants to use their preferred protocols and instruments to process and sequence the samples and controls. By uploading their sequencing data and associated metadata to Mosaic, participants will be able to compare the differences in their protocols to other participants and see the differences in taxonomic profiles. All of the data collected for this Challenge will be open source for the community to access and analyze, which will trigger a bioinformatics phase to the Challenge at a later stage.

The Standards Challenge is made possible through a collaboration between Janssen Research & Development, LLC, National Institute of Standards and Technology , The BioCollective, DNA Genotek Inc. , and DNAnexus.

  1. Provide the community with a diverse sample set that will allow an unprecedented number of labs around the world to compare their data generation pipelines.
  2. Provide Challenge participants with an interactive platform to compare their protocols and data produced from these samples to those of other Challenge participants.
  3. Provide the community with a unique data set that will allow any computational team to compare their analytical pipelines.
  4. Engage the community in discussions about standards and reproducibility of NGS-based microbiome profiling methods.
  • Developing a statistically robust model for variability and standard development which entails using fixed protocols
  • Developing (one) standard protocol(s)
  • Developing calibration standard(s) for NGS-based experiments

You should participate if you:

  • Are a microbiome lab using any NGS-based approaches (16S, whole genome shotgun, HiC, etc.) for microbiome profiling
  • Want to compare how your internal protocols differ from those of other participating labs using a highly characterized set of samples
  • Want to compare how the results from across participating labs differ due to the variety of protocols used
  • Are a computational lab or group working on metagenomics or microbiome analyses
  • Want to access raw sequencing data and protocol metadata to unravel the impact of different protocols on profiling results

Join the Challenge in 3 Steps!

  1. Create a Mosaic account and receive $50 worth of compute credits if you do not have a DNAnexus account yet.
  2. Request a set of samples
  3. Upload the reads and metadata to see your results!

See detailed instructions here.

Each set of 7 aliquots contains two different types of samples:
  1. 5 tubes of homogenized human fecal samples – Our partners at The BioCollective have prepared homogenized fecal samples from 5 human donors using DNA Genotek’s OMNIGene•Gut Solution and partitioned each sample into 700 aliquots.
  2. 2 tubes of microbial DNA reference material – The National Institute of Standards and Technology (NIST) has also provided 2 samples of microbial DNA reference material, mixed in known stoichiometries.
See Samples Description for more details.

Participants can begin to request samples on May 28th.

An overview presentation was delivered at the 20th Genomic Standards Consortium Meeting (May 2-4, 2018).

Challenge closes two months after all 700 sets of samples have been shipped out to Challenge participants.

All protocol metadata, sequencing data, analytical pipelines and results pertaining to the Standards Challenge that has been expressly shared, except confidential information regarding proprietary knowhow belonging to Challenge participants, The BioCollective, DNA Genotek and donor information, will be made publicly available via Mosaic.

We are planning to publish the results of this community effort in a top-ranked journal together with all participants who submit sequencing and protocol data as well as those who are providing apps and analyses.

Challenge Hosts
Powered By