Alternatives to Spark Framework logo

Alternatives to Spark Framework

Apache Spark, Hadoop, Spring, Spring Boot, and ExpressJS are the most popular alternatives and competitors to Spark Framework.
29
59
+ 1
7

What is Spark Framework and what are its top alternatives?

It is a simple and expressive Java/Kotlin web framework DSL built for rapid development. Its intention is to provide an alternative for Kotlin/Java developers that want to develop their web applications as expressive as possible and with minimal boilerplate.
Spark Framework is a tool in the Microframeworks (Backend) category of a tech stack.
Spark Framework is an open source tool with 9K GitHub stars and 1.5K GitHub forks. Here’s a link to Spark Framework's open source repository on GitHub

Top Alternatives to Spark Framework

  • Apache Spark

    Apache Spark

    Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning. ...

  • Hadoop

    Hadoop

    The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. ...

  • Spring

    Spring

    A key element of Spring is infrastructural support at the application level: Spring focuses on the "plumbing" of enterprise applications so that teams can focus on application-level business logic, without unnecessary ties to specific deployment environments. ...

  • Spring Boot

    Spring Boot

    Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can "just run". We take an opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration. ...

  • ExpressJS

    ExpressJS

    Express is a minimal and flexible node.js web application framework, providing a robust set of features for building single and multi-page, and hybrid web applications. ...

  • Flask

    Flask

    Flask is intended for getting started very quickly and was developed with best intentions in mind. ...

  • Django REST framework

    Django REST framework

    It is a powerful and flexible toolkit that makes it easy to build Web APIs.

  • Sinatra

    Sinatra

    Sinatra is a DSL for quickly creating web applications in Ruby with minimal effort. ...

Spark Framework alternatives & related posts

Apache Spark logo

Apache Spark

2.1K
2.4K
131
Fast and general engine for large-scale data processing
2.1K
2.4K
+ 1
131
PROS OF APACHE SPARK
  • 58
    Open-source
  • 47
    Fast and Flexible
  • 7
    One platform for every big data problem
  • 6
    Easy to install and to use
  • 6
    Great for distributed SQL like applications
  • 3
    Works well for most Datascience usecases
  • 2
    Machine learning libratimery, Streaming in real
  • 2
    In memory Computation
  • 0
    Interactive Query
CONS OF APACHE SPARK
  • 3
    Speed

related Apache Spark posts

Eric Colson
Chief Algorithms Officer at Stitch Fix · | 20 upvotes · 1.7M views

The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.

Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).

At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.

For more info:

#DataScience #DataStack #Data

See more
Conor Myhrvold
Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 876.6K views

Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

(Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

See more
Hadoop logo

Hadoop

1.9K
1.9K
54
Open-source software for reliable, scalable, distributed computing
1.9K
1.9K
+ 1
54
PROS OF HADOOP
  • 38
    Great ecosystem
  • 11
    One stack to rule them all
  • 4
    Great load balancer
  • 1
    Java syntax
CONS OF HADOOP
    Be the first to leave a con

    related Hadoop posts

    Conor Myhrvold
    Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 876.6K views

    Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

    Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

    https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

    (Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

    See more
    Shared insights
    on
    KafkaKafkaHadoopHadoop
    at

    The early data ingestion pipeline at Pinterest used Kafka as the central message transporter, with the app servers writing messages directly to Kafka, which then uploaded log files to S3.

    For databases, a custom Hadoop streamer pulled database data and wrote it to S3.

    Challenges cited for this infrastructure included high operational overhead, as well as potential data loss occurring when Kafka broker outages led to an overflow of in-memory message buffering.

    See more
    Spring logo

    Spring

    3K
    3.3K
    1K
    Provides a comprehensive programming and configuration model for modern Java-based enterprise applications
    3K
    3.3K
    + 1
    1K
    PROS OF SPRING
    • 216
      Java
    • 153
      Open source
    • 131
      Great community
    • 117
      Very powerful
    • 110
      Enterprise
    • 61
      Lot of great subprojects
    • 58
      Easy setup
    • 44
      Convention , configuration, done
    • 37
      Standard
    • 29
      Love the logic
    • 10
      Dependency injection
    • 10
      Good documentation
    • 9
      Stability
    • 6
      MVC
    • 6
      Easy
    • 3
      Makes the hard stuff fun & the easy stuff automatic
    • 3
      Strong typing
    • 2
      Great Desgin
    • 2
      Integrations with most other Java frameworks
    • 2
      Easy Integration with Spring Security
    • 2
      Maven
    • 1
      Best practices
    • 1
      Live project
    • 1
      OracleDb integration
    • 1
      Code maintenance
    • 1
      Large ecosystem with seamless integration
    • 1
      Java has more support and more libraries
    • 1
      Supports vast databases
    CONS OF SPRING
    • 12
      Draws you into its own ecosystem and bloat
    • 2
      Verbose configuration
    • 2
      Poor documentation
    • 1
      Java

    related Spring posts

    Is learning Spring and Spring Boot for web apps back-end development is still relevant in 2021? Feel free to share your views with comparison to Django/Node.js/ ExpressJS or other frameworks.

    Please share some good beginner resources to start learning about spring/spring boot framework to build the web apps.

    See more

    I am consulting for a company that wants to move its current CubeCart e-commerce site to another PHP based platform like PrestaShop or Magento. I was interested in alternatives that utilize Node.js as the primary platform. I currently don't know PHP, but I have done full stack dev with Java, Spring, Thymeleaf, etc.. I am just unsure that learning a set of technologies not commonly used makes sense. For example, in PrestaShop, I would need to work with JavaScript better and learn PHP, Twig, and Bootstrap. It seems more cumbersome than a Node JS system, where the language syntax stays the same for the full stack. I am looking for thoughts and advice on the relevance of PHP skillset into the future AND whether the Node based e-commerce open source options can compete with Magento or Prestashop.

    See more
    Spring Boot logo

    Spring Boot

    13.8K
    11.7K
    841
    Create Spring-powered, production-grade applications and services with absolute minimum fuss
    13.8K
    11.7K
    + 1
    841
    PROS OF SPRING BOOT
    • 127
      Powerful and handy
    • 121
      Easy setup
    • 111
      Java
    • 83
      Spring
    • 79
      Fast
    • 39
      Extensible
    • 32
      Lots of "off the shelf" functionalities
    • 27
      Cloud Solid
    • 21
      Caches well
    • 19
      Many receipes around for obscure features
    • 18
      Modular
    • 18
      Productive
    • 17
      Integrations with most other Java frameworks
    • 16
      Spring ecosystem is great
    • 16
      Fast Performance With Microservices
    • 14
      Community
    • 13
      Auto-configuration
    • 11
      Easy setup, Community Support, Solid for ERP apps
    • 11
      One-stop shop
    • 10
      Easy to parallelize
    • 9
      Cross-platform
    • 9
      Easy setup, good for build erp systems, well documented
    • 8
      Easy setup, Git Integration
    • 8
      Powerful 3rd party libraries and frameworks
    • 2
      Kotlin
    • 2
      It's so easier to start a project on spring
    CONS OF SPRING BOOT
    • 18
      Heavy weight
    • 17
      Annotation ceremony
    • 10
      Many config files needed
    • 7
      Java
    • 5
      Reactive
    • 4
      Excellent tools for cloud hosting, since 5.x

    related Spring Boot posts

    Praveen Mooli
    Engineering Manager at Taylor and Francis · | 14 upvotes · 1.6M views

    We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

    To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

    To build #Webapps we decided to use Angular 2 with RxJS

    #Devops - GitHub , Travis CI , Terraform , Docker , Serverless

    See more

    Is learning Spring and Spring Boot for web apps back-end development is still relevant in 2021? Feel free to share your views with comparison to Django/Node.js/ ExpressJS or other frameworks.

    Please share some good beginner resources to start learning about spring/spring boot framework to build the web apps.

    See more
    ExpressJS logo

    ExpressJS

    16.7K
    12.6K
    1.5K
    Sinatra inspired web development framework for node.js -- insanely fast, flexible, and simple
    16.7K
    12.6K
    + 1
    1.5K
    PROS OF EXPRESSJS
    • 362
      Simple
    • 318
      Node.js
    • 234
      Javascript
    • 183
      High performance
    • 147
      Robust routing
    • 66
      Open source
    • 63
      Middlewares
    • 51
      Great community
    • 33
      Hybrid web applications
    • 8
      Sinatra inspired
    • 8
      Well documented
    • 4
      Isomorphic js.. superfast and easy
    • 3
      Rapid development
    • 2
      Event loop
    • 2
      Socket connection
    • 2
      Npm
    • 2
      Resource available for learning
    • 2
      Light weight
    • 1
      Data stream
    • 1
      Callbacks
    • 0
      Xxx
    CONS OF EXPRESSJS
    • 21
      Not python
    • 15
      Overrated
    • 14
      No multithreading
    • 6
      Javascript
    • 4
      Not fast
    • 2
      Easily Insecure for Novices

    related ExpressJS posts

    Simon Reymann
    Senior Fullstack Developer at QUANTUSflow Software GmbH · | 23 upvotes · 1.5M views

    Our whole Node.js backend stack consists of the following tools:

    • Lerna as a tool for multi package and multi repository management
    • npm as package manager
    • NestJS as Node.js framework
    • TypeScript as programming language
    • ExpressJS as web server
    • Swagger UI for visualizing and interacting with the API’s resources
    • Postman as a tool for API development
    • TypeORM as object relational mapping layer
    • JSON Web Token for access token management

    The main reason we have chosen Node.js over PHP is related to the following artifacts:

    • Made for the web and widely in use: Node.js is a software platform for developing server-side network services. Well-known projects that rely on Node.js include the blogging software Ghost, the project management tool Trello and the operating system WebOS. Node.js requires the JavaScript runtime environment V8, which was specially developed by Google for the popular Chrome browser. This guarantees a very resource-saving architecture, which qualifies Node.js especially for the operation of a web server. Ryan Dahl, the developer of Node.js, released the first stable version on May 27, 2009. He developed Node.js out of dissatisfaction with the possibilities that JavaScript offered at the time. The basic functionality of Node.js has been mapped with JavaScript since the first version, which can be expanded with a large number of different modules. The current package managers (npm or Yarn) for Node.js know more than 1,000,000 of these modules.
    • Fast server-side solutions: Node.js adopts the JavaScript "event-loop" to create non-blocking I/O applications that conveniently serve simultaneous events. With the standard available asynchronous processing within JavaScript/TypeScript, highly scalable, server-side solutions can be realized. The efficient use of the CPU and the RAM is maximized and more simultaneous requests can be processed than with conventional multi-thread servers.
    • A language along the entire stack: Widely used frameworks such as React or AngularJS or Vue.js, which we prefer, are written in JavaScript/TypeScript. If Node.js is now used on the server side, you can use all the advantages of a uniform script language throughout the entire application development. The same language in the back- and frontend simplifies the maintenance of the application and also the coordination within the development team.
    • Flexibility: Node.js sets very few strict dependencies, rules and guidelines and thus grants a high degree of flexibility in application development. There are no strict conventions so that the appropriate architecture, design structures, modules and features can be freely selected for the development.
    See more

    Hello, I hope everyone is doing good and safe. I need advice on what to learn more, I have started learning HTML, CSS, Bootstrap, JavaScript, Node.js, ExpressJS, React. eventually will learn MongoDB too. I would like to be a Front End developer or full-stack developer. What else would be the suggestion to get a job and what things I need to focus more on as a fresher to make my skills better. Do I have to be good in Algorithms and Dynamic Programming to find a job for entry-level? Looking forward to hearing from you guys for suggestions. 

    See more
    Flask logo

    Flask

    12.2K
    9.9K
    1.4K
    A microframework for Python based on Werkzeug, Jinja 2 and good intentions
    12.2K
    9.9K
    + 1
    1.4K
    PROS OF FLASK
    • 298
      Lightweight
    • 257
      Python
    • 207
      Minimal
    • 140
      Open source
    • 95
      Documentation
    • 62
      Easy to use
    • 51
      Easy to setup and get it going
    • 51
      Well designed
    • 45
      Easy to develop and maintain applications
    • 43
      Easy to get started
    • 15
      Beautiful code
    • 14
      Rapid development
    • 12
      Expressive
    • 12
      Powerful
    • 11
      Awesome
    • 10
      Love it
    • 10
      Speed
    • 9
      Simple to use
    • 9
      Flexibilty
    • 8
      Get started quickly
    • 8
      For it flexibility
    • 8
      Perfect for small to large projects with superb docs.
    • 7
      Flexibilty and easy to use
    • 7
      Easy to integrate
    • 7
      Productive
    • 6
      Customizable
    • 6
      Not JS
    • 5
      Secured
    • 5
      User friendly
    • 5
      Flask
    • 3
      Unopinionated
    CONS OF FLASK
    • 10
      Not JS
    • 7
      Context
    • 3
      Not fast

    related Flask posts

    James Man
    Software Engineer at Pinterest · | 39 upvotes · 710.7K views
    Shared insights
    on
    FlaskFlaskReactReact
    at

    One of our top priorities at Pinterest is fostering a safe and trustworthy experience for all Pinners. As Pinterest’s user base and ads business grow, the review volume has been increasing exponentially, and more content types require moderation support. To solve greater engineering and operational challenges at scale, we needed a highly-reliable and performant system to detect, report, evaluate, and act on abusive content and users and so we created Pinqueue.

    Pinqueue-3.0 serves as a generic platform for content moderation and human labeling. Under the hood, Pinqueue3.0 is a Flask + React app powered by Pinterest’s very own Gestalt UI framework. On the backend, Pinqueue3.0 heavily relies on PinLater, a Pinterest-built reliable asynchronous job execution system, to handle the requests for enqueueing and action-taking. Using PinLater has significantly strengthened Pinqueue3.0’s overall infra with its capability of processing a massive load of events with configurable retry policies.

    Hundreds of millions of people around the world use Pinterest to discover and do what they love, and our job is to protect them from abusive and harmful content. We’re committed to providing an inspirational yet safe experience to all Pinners. Solving trust & safety problems is a joint effort requiring expertise across multiple domains. Pinqueue3.0 not only plays a critical role in responsively taking down unsafe content, it also has become an enabler for future ML/automation initiatives by providing high-quality human labels. Going forward, we will continue to improve the review experience, measure review quality and collaborate with our machine learning teams to solve content moderation beyond manual reviews at an even larger scale.

    See more

    Hey, so I developed a basic application with Python. But to use it, you need a python interpreter. I want to add a GUI to make it more appealing. What should I choose to develop a GUI? I have very basic skills in front end development (CSS, JavaScript). I am fluent in python. I'm looking for a tool that is easy to use and doesn't require too much code knowledge. I have recently tried out Flask, but it is kinda complicated. Should I stick with it, move to Django, or is there another nice framework to use?

    See more
    Django REST framework logo

    Django REST framework

    1.4K
    1.4K
    297
    Web APIs for Django
    1.4K
    1.4K
    + 1
    297
    PROS OF DJANGO REST FRAMEWORK
    • 63
      Browsable api
    • 62
      Easy to use
    • 52
      Great documentation
    • 49
      Customizable
    • 41
      Fast development
    • 9
      Easy to use, customizable, pluggable, serializer
    • 8
      Python
    • 5
      Django ORM
    • 4
      FastSerialize
    • 2
      Easy implementation
    • 2
      Less code
    • 0
      Dsasda
    CONS OF DJANGO REST FRAMEWORK
    • 2
      Bad documentation
    • 2
      Reimplements Django functionality
    • 1
      No support for URL Namespaces
    • 0
      Bad CSRF handling

    related Django REST framework posts

    Tim Abbott

    Zulip has been powered by Django since the very early days of its development with Django 1.4, back in 2012. As a reasonably mature web application with significant scale, we're at the stage in many companies' development where one starts to rip out more and more of the web framework to optimize things or just make them work the way we want. (E.g. while I was at Dropbox in early 2016, we discovered we only had about 600 lines of code left from the original Pylons framework that actually ran).

    One of the things that has been really fantastic about Django is that we're still happily using it for the vast majority of code in the project, and every time Django comes out with a new release, I read the changelog and get excited about several improvements that actually make my life better. While Django has made some design decisions that I don't agree with (e.g. I'm not a fan of Django REST framework, and think it makes life more difficult), Django also makes it easy to do your own thing, which we've done to great effect (see the linked article for details on our has_request_variables framework).

    Overall I think we've gotten a ton of value out of Python and Django and would recommend it to anyone starting a new full-featured web application project today.

    See more

    Hi

    I’ve been using Django for the last year on and off to do my backend API. I’m getting a bit frustrated with the Django REST framework with the setup of the serializers and Django for the lack of web sockets. I’m considering either Spring or .NET Core. I’m familiar with Kotlin and C# but I’ve not built any substantial projects with them. I like OOP, building a desktop app, web API, and also the potential to get a job in the future or building a tool at work to manage my documents, dashboard and processes point cloud data.

    I’m familiar with c/cpp, TypeScript.

    I would love your insights on where I should go.

    See more
    Sinatra logo

    Sinatra

    559
    431
    211
    Classy web-development dressed in a DSL
    559
    431
    + 1
    211
    PROS OF SINATRA
    • 65
      Lightweight
    • 49
      Simple
    • 35
      Open source
    • 20
      Ruby
    • 13
      Great ecosystem of tools
    • 10
      Ease of use
    • 8
      If you know http you know sinatra
    • 5
      Fast
    • 5
      Large Community
    • 1
      Flexibilty and easy to use
    CONS OF SINATRA
      Be the first to leave a con

      related Sinatra posts