DeepSpeed vs Manifold

DeepSpeed vs Manifold: What are the differences?

DeepSpeed: A deep learning optimization library that makes distributed training easy, efficient, and effective (By Microsoft). It is a deep learning optimization library that makes distributed training easy, efficient, and effective. It can train DL models with over a hundred billion parameters on the current generation of GPU clusters while achieving over 5x in system performance compared to the state-of-art. Early adopters of DeepSpeed have already produced a language model (LM) with over 17B parameters called Turing-NLG, establishing a new SOTA in the LM category; Manifold: A model-agnostic visual debugging tool for machine learning. Understanding ML model performance and behavior is a non-trivial process, given the intrisic opacity of ML algorithms. Performance summary statistics such as AUC, RMSE, and others are not instructive enough for identifying what went wrong with a model or how to improve it. As a visual analytics tool, Manifold allows ML practitioners to look beyond overall summary metrics to detect which subset of data a model is inaccurately predicting.

DeepSpeed and Manifold can be primarily classified as "Machine Learning" tools.

Some of the features offered by DeepSpeed are:

• Distributed Training with Mixed Precision
• Model Parallelism
• Memory and Bandwidth Optimizations

On the other hand, Manifold provides the following key features:

• Performance Comparison View
• Feature Attribution View
• Histogram / heatmap

DeepSpeed and Manifold are both open source tools. DeepSpeed with 1.98K GitHub stars and 134 forks on GitHub appears to be more popular than Manifold with 1.06K GitHub stars and 78 GitHub forks.