Alternatives to Spark Framework logo

Alternatives to Spark Framework

Apache Spark, Hadoop, Spring, Spring Boot, and ExpressJS are the most popular alternatives and competitors to Spark Framework.
40
91
+ 1
7

What is Spark Framework and what are its top alternatives?

It is a simple and expressive Java/Kotlin web framework DSL built for rapid development. Its intention is to provide an alternative for Kotlin/Java developers that want to develop their web applications as expressive as possible and with minimal boilerplate.
Spark Framework is a tool in the Microframeworks (Backend) category of a tech stack.
Spark Framework is an open source tool with 9.6K GitHub stars and 1.6K GitHub forks. Here’s a link to Spark Framework's open source repository on GitHub

Top Alternatives to Spark Framework

  • Apache Spark
    Apache Spark

    Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning. ...

  • Hadoop
    Hadoop

    The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. ...

  • Spring
    Spring

    A key element of Spring is infrastructural support at the application level: Spring focuses on the "plumbing" of enterprise applications so that teams can focus on application-level business logic, without unnecessary ties to specific deployment environments. ...

  • Spring Boot
    Spring Boot

    Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can "just run". We take an opinionated view of the Spring platform and third-party libraries so you can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration. ...

  • ExpressJS
    ExpressJS

    Express is a minimal and flexible node.js web application framework, providing a robust set of features for building single and multi-page, and hybrid web applications. ...

  • Flask
    Flask

    Flask is intended for getting started very quickly and was developed with best intentions in mind. ...

  • Django REST framework
    Django REST framework

    It is a powerful and flexible toolkit that makes it easy to build Web APIs.

  • Sinatra
    Sinatra

    Sinatra is a DSL for quickly creating web applications in Ruby with minimal effort. ...

Spark Framework alternatives & related posts

Apache Spark logo

Apache Spark

3K
3.5K
140
Fast and general engine for large-scale data processing
3K
3.5K
+ 1
140
PROS OF APACHE SPARK
  • 61
    Open-source
  • 48
    Fast and Flexible
  • 8
    One platform for every big data problem
  • 8
    Great for distributed SQL like applications
  • 6
    Easy to install and to use
  • 3
    Works well for most Datascience usecases
  • 2
    Interactive Query
  • 2
    Machine learning libratimery, Streaming in real
  • 2
    In memory Computation
CONS OF APACHE SPARK
  • 4
    Speed

related Apache Spark posts

Eric Colson
Chief Algorithms Officer at Stitch Fix · | 21 upvotes · 6.1M views

The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.

Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).

At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.

For more info:

#DataScience #DataStack #Data

See more
Conor Myhrvold
Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 2.9M views

Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

(Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

See more
Hadoop logo

Hadoop

2.7K
2.3K
56
Open-source software for reliable, scalable, distributed computing
2.7K
2.3K
+ 1
56
PROS OF HADOOP
  • 39
    Great ecosystem
  • 11
    One stack to rule them all
  • 4
    Great load balancer
  • 1
    Amazon aws
  • 1
    Java syntax
CONS OF HADOOP
    Be the first to leave a con

    related Hadoop posts

    Shared insights
    on
    KafkaKafkaHadoopHadoop
    at

    The early data ingestion pipeline at Pinterest used Kafka as the central message transporter, with the app servers writing messages directly to Kafka, which then uploaded log files to S3.

    For databases, a custom Hadoop streamer pulled database data and wrote it to S3.

    Challenges cited for this infrastructure included high operational overhead, as well as potential data loss occurring when Kafka broker outages led to an overflow of in-memory message buffering.

    See more
    Conor Myhrvold
    Tech Brand Mgr, Office of CTO at Uber · | 7 upvotes · 2.9M views

    Why we built Marmaray, an open source generic data ingestion and dispersal framework and library for Apache Hadoop :

    Built and designed by our Hadoop Platform team, Marmaray is a plug-in-based framework built on top of the Hadoop ecosystem. Users can add support to ingest data from any source and disperse to any sink leveraging the use of Apache Spark . The name, Marmaray, comes from a tunnel in Turkey connecting Europe and Asia. Similarly, we envisioned Marmaray within Uber as a pipeline connecting data from any source to any sink depending on customer preference:

    https://eng.uber.com/marmaray-hadoop-ingestion-open-source/

    (Direct GitHub repo: https://github.com/uber/marmaray Kafka Kafka Manager )

    See more
    Spring logo

    Spring

    3.9K
    4.7K
    1.1K
    Provides a comprehensive programming and configuration model for modern Java-based enterprise applications
    3.9K
    4.7K
    + 1
    1.1K
    PROS OF SPRING
    • 230
      Java
    • 157
      Open source
    • 136
      Great community
    • 123
      Very powerful
    • 114
      Enterprise
    • 64
      Lot of great subprojects
    • 60
      Easy setup
    • 44
      Convention , configuration, done
    • 40
      Standard
    • 31
      Love the logic
    • 13
      Good documentation
    • 11
      Dependency injection
    • 11
      Stability
    • 9
      MVC
    • 6
      Easy
    • 3
      Makes the hard stuff fun & the easy stuff automatic
    • 3
      Strong typing
    • 2
      Code maintenance
    • 2
      Best practices
    • 2
      Maven
    • 2
      Great Desgin
    • 2
      Easy Integration with Spring Security
    • 2
      Integrations with most other Java frameworks
    • 1
      Java has more support and more libraries
    • 1
      Supports vast databases
    • 1
      Large ecosystem with seamless integration
    • 1
      OracleDb integration
    • 1
      Live project
    CONS OF SPRING
    • 15
      Draws you into its own ecosystem and bloat
    • 3
      Verbose configuration
    • 3
      Poor documentation
    • 3
      Java
    • 2
      Java is more verbose language in compare to python

    related Spring posts

    Is learning Spring and Spring Boot for web apps back-end development is still relevant in 2021? Feel free to share your views with comparison to Django/Node.js/ ExpressJS or other frameworks.

    Please share some good beginner resources to start learning about spring/spring boot framework to build the web apps.

    See more

    I am consulting for a company that wants to move its current CubeCart e-commerce site to another PHP based platform like PrestaShop or Magento. I was interested in alternatives that utilize Node.js as the primary platform. I currently don't know PHP, but I have done full stack dev with Java, Spring, Thymeleaf, etc.. I am just unsure that learning a set of technologies not commonly used makes sense. For example, in PrestaShop, I would need to work with JavaScript better and learn PHP, Twig, and Bootstrap. It seems more cumbersome than a Node JS system, where the language syntax stays the same for the full stack. I am looking for thoughts and advice on the relevance of PHP skillset into the future AND whether the Node based e-commerce open source options can compete with Magento or Prestashop.

    See more
    Spring Boot logo

    Spring Boot

    25.3K
    22.9K
    1K
    Create Spring-powered, production-grade applications and services with absolute minimum fuss
    25.3K
    22.9K
    + 1
    1K
    PROS OF SPRING BOOT
    • 149
      Powerful and handy
    • 134
      Easy setup
    • 128
      Java
    • 90
      Spring
    • 85
      Fast
    • 46
      Extensible
    • 37
      Lots of "off the shelf" functionalities
    • 32
      Cloud Solid
    • 26
      Caches well
    • 24
      Productive
    • 24
      Many receipes around for obscure features
    • 23
      Modular
    • 23
      Integrations with most other Java frameworks
    • 22
      Spring ecosystem is great
    • 21
      Auto-configuration
    • 21
      Fast Performance With Microservices
    • 18
      Community
    • 17
      Easy setup, Community Support, Solid for ERP apps
    • 15
      One-stop shop
    • 14
      Easy to parallelize
    • 14
      Cross-platform
    • 13
      Easy setup, good for build erp systems, well documented
    • 13
      Powerful 3rd party libraries and frameworks
    • 12
      Easy setup, Git Integration
    • 5
      It's so easier to start a project on spring
    • 4
      Kotlin
    • 1
      Microservice and Reactive Programming
    • 1
      The ability to integrate with the open source ecosystem
    CONS OF SPRING BOOT
    • 23
      Heavy weight
    • 18
      Annotation ceremony
    • 13
      Java
    • 11
      Many config files needed
    • 5
      Reactive
    • 4
      Excellent tools for cloud hosting, since 5.x
    • 1
      Java 😒😒

    related Spring Boot posts

    Praveen Mooli
    Engineering Manager at Taylor and Francis · | 18 upvotes · 3.8M views

    We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

    To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

    To build #Webapps we decided to use Angular 2 with RxJS

    #Devops - GitHub , Travis CI , Terraform , Docker , Serverless

    See more

    Is learning Spring and Spring Boot for web apps back-end development is still relevant in 2021? Feel free to share your views with comparison to Django/Node.js/ ExpressJS or other frameworks.

    Please share some good beginner resources to start learning about spring/spring boot framework to build the web apps.

    See more
    ExpressJS logo

    ExpressJS

    33.8K
    23.2K
    1.6K
    Sinatra inspired web development framework for node.js -- insanely fast, flexible, and simple
    33.8K
    23.2K
    + 1
    1.6K
    PROS OF EXPRESSJS
    • 380
      Simple
    • 336
      Node.js
    • 244
      Javascript
    • 193
      High performance
    • 152
      Robust routing
    • 73
      Middlewares
    • 71
      Open source
    • 59
      Great community
    • 37
      Hybrid web applications
    • 16
      Well documented
    • 9
      Sinatra inspired
    • 9
      Rapid development
    • 7
      Socket connection
    • 7
      Isomorphic js.. superfast and easy
    • 5
      Light weight
    • 4
      Resource available for learning
    • 4
      Npm
    • 3
      Event loop
    • 3
      Callbacks
    • 2
      Data stream
    CONS OF EXPRESSJS
    • 27
      Not python
    • 17
      Overrated
    • 14
      No multithreading
    • 9
      Javascript
    • 5
      Not fast
    • 2
      Easily Insecure for Novices

    related ExpressJS posts

    Simon Reymann
    Senior Fullstack Developer at QUANTUSflow Software GmbH · | 27 upvotes · 4.7M views

    Our whole Node.js backend stack consists of the following tools:

    • Lerna as a tool for multi package and multi repository management
    • npm as package manager
    • NestJS as Node.js framework
    • TypeScript as programming language
    • ExpressJS as web server
    • Swagger UI for visualizing and interacting with the API’s resources
    • Postman as a tool for API development
    • TypeORM as object relational mapping layer
    • JSON Web Token for access token management

    The main reason we have chosen Node.js over PHP is related to the following artifacts:

    • Made for the web and widely in use: Node.js is a software platform for developing server-side network services. Well-known projects that rely on Node.js include the blogging software Ghost, the project management tool Trello and the operating system WebOS. Node.js requires the JavaScript runtime environment V8, which was specially developed by Google for the popular Chrome browser. This guarantees a very resource-saving architecture, which qualifies Node.js especially for the operation of a web server. Ryan Dahl, the developer of Node.js, released the first stable version on May 27, 2009. He developed Node.js out of dissatisfaction with the possibilities that JavaScript offered at the time. The basic functionality of Node.js has been mapped with JavaScript since the first version, which can be expanded with a large number of different modules. The current package managers (npm or Yarn) for Node.js know more than 1,000,000 of these modules.
    • Fast server-side solutions: Node.js adopts the JavaScript "event-loop" to create non-blocking I/O applications that conveniently serve simultaneous events. With the standard available asynchronous processing within JavaScript/TypeScript, highly scalable, server-side solutions can be realized. The efficient use of the CPU and the RAM is maximized and more simultaneous requests can be processed than with conventional multi-thread servers.
    • A language along the entire stack: Widely used frameworks such as React or AngularJS or Vue.js, which we prefer, are written in JavaScript/TypeScript. If Node.js is now used on the server side, you can use all the advantages of a uniform script language throughout the entire application development. The same language in the back- and frontend simplifies the maintenance of the application and also the coordination within the development team.
    • Flexibility: Node.js sets very few strict dependencies, rules and guidelines and thus grants a high degree of flexibility in application development. There are no strict conventions so that the appropriate architecture, design structures, modules and features can be freely selected for the development.
    See more

    Repost

    Overview: To put it simply, we plan to use the MERN stack to build our web application. MongoDB will be used as our primary database. We will use ExpressJS alongside Node.js to set up our API endpoints. Additionally, we plan to use React to build our SPA on the client side and use Redis on the server side as our primary caching solution. Initially, while working on the project, we plan to deploy our server and client both on Heroku . However, Heroku is very limited and we will need the benefits of an Infrastructure as a Service so we will use Amazon EC2 to later deploy our final version of the application.

    Serverside: nodemon will allow us to automatically restart a running instance of our node app when files changes take place. We decided to use MongoDB because it is a non relational database which uses the Document Object Model. This allows a lot of flexibility as compared to a RDMS like SQL which requires a very structural model of data that does not change too much. Another strength of MongoDB is its ease in scalability. We will use Mongoose along side MongoDB to model our application data. Additionally, we will host our MongoDB cluster remotely on MongoDB Atlas. Bcrypt will be used to encrypt user passwords that will be stored in the DB. This is to avoid the risks of storing plain text passwords. Moreover, we will use Cloudinary to store images uploaded by the user. We will also use the Twilio SendGrid API to enable automated emails sent by our application. To protect private API endpoints, we will use JSON Web Token and Passport. Also, PayPal will be used as a payment gateway to accept payments from users.

    Client Side: As mentioned earlier, we will use React to build our SPA. React uses a virtual DOM which is very efficient in rendering a page. Also React will allow us to reuse components. Furthermore, it is very popular and there is a large community that uses React so it can be helpful if we run into issues. We also plan to make a cross platform mobile application later and using React will allow us to reuse a lot of our code with React Native. Redux will be used to manage state. Redux works great with React and will help us manage a global state in the app and avoid the complications of each component having its own state. Additionally, we will use Bootstrap components and custom CSS to style our app.

    Other: Git will be used for version control. During the later stages of our project, we will use Google Analytics to collect useful data regarding user interactions. Moreover, Slack will be our primary communication tool. Also, we will use Visual Studio Code as our primary code editor because it is very light weight and has a wide variety of extensions that will boost productivity. Postman will be used to interact with and debug our API endpoints.

    See more
    Flask logo

    Flask

    18.9K
    15.8K
    82
    A microframework for Python based on Werkzeug, Jinja 2 and good intentions
    18.9K
    15.8K
    + 1
    82
    PROS OF FLASK
    • 14
      Flexibilty
    • 10
      For it flexibility
    • 9
      Flexibilty and easy to use
    • 8
      Flask
    • 7
      User friendly
    • 6
      Secured
    • 5
      Unopinionated
    • 3
      Orm
    • 2
      Secure
    • 1
      Beautiful code
    • 1
      Easy to get started
    • 1
      Easy to develop and maintain applications
    • 1
      Not JS
    • 1
      Easy to use
    • 1
      Documentation
    • 1
      Python
    • 1
      Minimal
    • 1
      Lightweight
    • 1
      Easy to setup and get it going
    • 1
      Perfect for small to large projects with superb docs.
    • 1
      Easy to integrate
    • 1
      Speed
    • 1
      Get started quickly
    • 1
      Customizable
    • 1
      Simple to use
    • 1
      Powerful
    • 1
      Rapid development
    • 0
      Open source
    • 0
      Well designed
    • 0
      Productive
    • 0
      Awesome
    • 0
      Expressive
    • 0
      Love it
    CONS OF FLASK
    • 10
      Not JS
    • 7
      Context
    • 5
      Not fast
    • 1
      Don't has many module as in spring

    related Flask posts

    James Man
    Software Engineer at Pinterest · | 45 upvotes · 2.7M views
    Shared insights
    on
    FlaskFlaskReactReact
    at

    One of our top priorities at Pinterest is fostering a safe and trustworthy experience for all Pinners. As Pinterest’s user base and ads business grow, the review volume has been increasing exponentially, and more content types require moderation support. To solve greater engineering and operational challenges at scale, we needed a highly-reliable and performant system to detect, report, evaluate, and act on abusive content and users and so we created Pinqueue.

    Pinqueue-3.0 serves as a generic platform for content moderation and human labeling. Under the hood, Pinqueue3.0 is a Flask + React app powered by Pinterest’s very own Gestalt UI framework. On the backend, Pinqueue3.0 heavily relies on PinLater, a Pinterest-built reliable asynchronous job execution system, to handle the requests for enqueueing and action-taking. Using PinLater has significantly strengthened Pinqueue3.0’s overall infra with its capability of processing a massive load of events with configurable retry policies.

    Hundreds of millions of people around the world use Pinterest to discover and do what they love, and our job is to protect them from abusive and harmful content. We’re committed to providing an inspirational yet safe experience to all Pinners. Solving trust & safety problems is a joint effort requiring expertise across multiple domains. Pinqueue3.0 not only plays a critical role in responsively taking down unsafe content, it also has become an enabler for future ML/automation initiatives by providing high-quality human labels. Going forward, we will continue to improve the review experience, measure review quality and collaborate with our machine learning teams to solve content moderation beyond manual reviews at an even larger scale.

    See more

    Hey, so I developed a basic application with Python. But to use it, you need a python interpreter. I want to add a GUI to make it more appealing. What should I choose to develop a GUI? I have very basic skills in front end development (CSS, JavaScript). I am fluent in python. I'm looking for a tool that is easy to use and doesn't require too much code knowledge. I have recently tried out Flask, but it is kinda complicated. Should I stick with it, move to Django, or is there another nice framework to use?

    See more
    Django REST framework logo

    Django REST framework

    1.9K
    2K
    310
    Web APIs for Django
    1.9K
    2K
    + 1
    310
    PROS OF DJANGO REST FRAMEWORK
    • 66
      Easy to use
    • 65
      Browsable api
    • 53
      Great documentation
    • 50
      Customizable
    • 42
      Fast development
    • 9
      Easy to use, customizable, pluggable, serializer
    • 8
      Python
    • 7
      Django ORM
    • 5
      FastSerialize
    • 3
      Less code
    • 2
      Easy implementation
    CONS OF DJANGO REST FRAMEWORK
    • 2
      Bad documentation
    • 2
      Reimplements Django functionality
    • 1
      No support for URL Namespaces
    • 0
      Bad CSRF handling

    related Django REST framework posts

    Arjiv Jivithkumar
    Junior Product Manager at Vozz · | 13 upvotes · 71.9K views

    Hey everyone! I'm planning on building a personal project - this will be my first full-stack project and will be a web app.

    The way it will work is that users will be able to post groups. This can be, groups for studying or groups for work, etc. They can also set the desired group size (e.g. limit the group to 3 members). Other users can then join said group - once the group is full, it will automatically close.

    What tech stack would you all recommend for this? I have a lot of experience with Django so maybe that will be good for the backend but I'm not sure where to go from there. I've heard using the Django REST framework with a React frontend might be good. Always open to learning new technologies and thanks in advance!

    See more

    I am planning on creating an application using the following tech-stack. Vue.js (TypeScript) for the front-end, Django (specifically Django REST framework) for the server-side work, and using PostgreSQL as the database. Is there any reason NOT to use this tech stack mentioned or are there better options? Without giving away too much info, my app will be logging information from the user, displaying this information, setting goals, displaying visual graphs, a friend system where you can add other people etc...

    See more
    Sinatra logo

    Sinatra

    1.1K
    498
    212
    Classy web-development dressed in a DSL
    1.1K
    498
    + 1
    212
    PROS OF SINATRA
    • 65
      Lightweight
    • 50
      Simple
    • 35
      Open source
    • 20
      Ruby
    • 13
      Great ecosystem of tools
    • 10
      Ease of use
    • 8
      If you know http you know sinatra
    • 5
      Large Community
    • 5
      Fast
    • 1
      Flexibilty and easy to use
    CONS OF SINATRA
      Be the first to leave a con

      related Sinatra posts