DiaWhat is Spark NLP?
It is a Natural Language Processing library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that scale easily in a distributed environment. It comes with 160+ pretrained pipelines and models in more than 20+ languages.
Spark NLP is a tool in the Text & Language Models category of a tech stack.
Key Features
TokenizationStop Words RemovalNormalizerStemmerLemmatizerNGramsRegex MatchingText MatchingChunkingDate MatcherPart-of-speech taggingSentence DetectorDependency parsing (Labeled/unlabled)Sentiment Detection (ML models)Spell Checker (ML and DL models)Word Embeddings (GloVe and Word2Vec)BERT EmbeddingsELMO EmbeddingsUniversal Sentence Encoder
Sentence EmbeddingsChunk Embeddings
Spark NLP Pros & Cons
Pros of Spark NLP
No pros listed yet.
Cons of Spark NLP
No cons listed yet.
Spark NLP Alternatives & Comparisons
What are some alternatives to Spark NLP?
Transformers
It provides general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in 100+ languages and deep interoperability between TensorFlow 2.0 and PyTorch.
SpaCy
It is a library for advanced Natural Language Processing in Python and Cython. It's built on the very latest research, and was designed from day one to be used in real products. It comes with pre-trained statistical models and word vectors, and currently supports tokenization for 49+ languages.
rasa NLU
rasa NLU (Natural Language Understanding) is a tool for intent classification and entity extraction. You can think of rasa NLU as a set of high level APIs for building your own language parser using existing NLP and ML libraries.
