Alternatives to YARN Hadoop logo

Alternatives to YARN Hadoop

Yarn, Apache Mesos, Nomad, DC/OS, and Mesosphere are the most popular alternatives and competitors to YARN Hadoop.
79
47
+ 1
1

What is YARN Hadoop and what are its top alternatives?

Its fundamental idea is to split up the functionalities of resource management and job scheduling/monitoring into separate daemons. The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM).
YARN Hadoop is a tool in the Cluster Management category of a tech stack.

Top Alternatives to YARN Hadoop

  • Yarn

    Yarn

    Yarn caches every package it downloads so it never needs to again. It also parallelizes operations to maximize resource utilization so install times are faster than ever. ...

  • Apache Mesos

    Apache Mesos

    Apache Mesos is a cluster manager that simplifies the complexity of running applications on a shared pool of servers. ...

  • Nomad

    Nomad

    Nomad is a cluster manager, designed for both long lived services and short lived batch processing workloads. Developers use a declarative job specification to submit work, and Nomad ensures constraints are satisfied and resource utilization is optimized by efficient task packing. Nomad supports all major operating systems and virtualized, containerized, or standalone applications. ...

  • DC/OS

    DC/OS

    Unlike traditional operating systems, DC/OS spans multiple machines within a network, aggregating their resources to maximize utilization by distributed applications. ...

  • Mesosphere

    Mesosphere

    Mesosphere offers a layer of software that organizes your machines, VMs, and cloud instances and lets applications draw from a single pool of intelligently- and dynamically-allocated resources, increasing efficiency and reducing operational complexity. ...

  • kops

    kops

    It helps you create, destroy, upgrade and maintain production-grade, highly available, Kubernetes clusters from the command line. AWS (Amazon Web Services) is currently officially supported, with GCE in beta support , and VMware vSphere in alpha, and other platforms planned. ...

  • Apache Aurora

    Apache Aurora

    Apache Aurora is a service scheduler that runs on top of Mesos, enabling you to run long-running services that take advantage of Mesos' scalability, fault-tolerance, and resource isolation. ...

  • Gardener

    Gardener

    Many Open Source tools exist which help in creating and updating single Kubernetes clusters. However, the more clusters you need the harder it becomes to operate, monitor, manage and keep all of them alive and up-to-date. And that is exactly what project Gardener focuses on. ...

YARN Hadoop alternatives & related posts

Yarn logo

Yarn

8.9K
5.9K
141
A new package manager for JavaScript
8.9K
5.9K
+ 1
141

related Yarn posts

Simon Reymann
Senior Fullstack Developer at QUANTUSflow Software GmbH · | 23 upvotes · 1.3M views

Our whole Node.js backend stack consists of the following tools:

  • Lerna as a tool for multi package and multi repository management
  • npm as package manager
  • NestJS as Node.js framework
  • TypeScript as programming language
  • ExpressJS as web server
  • Swagger UI for visualizing and interacting with the API’s resources
  • Postman as a tool for API development
  • TypeORM as object relational mapping layer
  • JSON Web Token for access token management

The main reason we have chosen Node.js over PHP is related to the following artifacts:

  • Made for the web and widely in use: Node.js is a software platform for developing server-side network services. Well-known projects that rely on Node.js include the blogging software Ghost, the project management tool Trello and the operating system WebOS. Node.js requires the JavaScript runtime environment V8, which was specially developed by Google for the popular Chrome browser. This guarantees a very resource-saving architecture, which qualifies Node.js especially for the operation of a web server. Ryan Dahl, the developer of Node.js, released the first stable version on May 27, 2009. He developed Node.js out of dissatisfaction with the possibilities that JavaScript offered at the time. The basic functionality of Node.js has been mapped with JavaScript since the first version, which can be expanded with a large number of different modules. The current package managers (npm or Yarn) for Node.js know more than 1,000,000 of these modules.
  • Fast server-side solutions: Node.js adopts the JavaScript "event-loop" to create non-blocking I/O applications that conveniently serve simultaneous events. With the standard available asynchronous processing within JavaScript/TypeScript, highly scalable, server-side solutions can be realized. The efficient use of the CPU and the RAM is maximized and more simultaneous requests can be processed than with conventional multi-thread servers.
  • A language along the entire stack: Widely used frameworks such as React or AngularJS or Vue.js, which we prefer, are written in JavaScript/TypeScript. If Node.js is now used on the server side, you can use all the advantages of a uniform script language throughout the entire application development. The same language in the back- and frontend simplifies the maintenance of the application and also the coordination within the development team.
  • Flexibility: Node.js sets very few strict dependencies, rules and guidelines and thus grants a high degree of flexibility in application development. There are no strict conventions so that the appropriate architecture, design structures, modules and features can be freely selected for the development.
See more
Johnny Bell
Software Engineer at Weedmaps · | 18 upvotes · 1M views

So when starting a new project you generally have your go to tools to get your site up and running locally, and some scripts to build out a production version of your site. Create React App is great for that, however for my projects I feel as though there is to much bloat in Create React App and if I use it, then I'm tied to React, which I love but if I want to switch it up to Vue or something I want that flexibility.

So to start everything up and running I clone my personal Webpack boilerplate - This is still in Webpack 3, and does need some updating but gets the job done for now. So given the name of the repo you may have guessed that yes I am using Webpack as my bundler I use Webpack because it is so powerful, and even though it has a steep learning curve once you get it, its amazing.

The next thing I do is make sure my machine has Node.js configured and the right version installed then run Yarn. I decided to use Yarn because when I was building out this project npm had some shortcomings such as no .lock file. I could probably move from Yarn to npm but I don't really see any point really.

I use Babel to transpile all of my #ES6 to #ES5 so the browser can read it, I love Babel and to be honest haven't looked up any other transpilers because Babel is amazing.

Finally when developing I have Prettier setup to make sure all my code is clean and uniform across all my JS files, and ESLint to make sure I catch any errors or code that could be optimized.

I'm really happy with this stack for my local env setup, and I'll probably stick with it for a while.

See more
Apache Mesos logo

Apache Mesos

276
348
30
Develop and run resource-efficient distributed systems
276
348
+ 1
30

related Apache Mesos posts

Docker containers on Mesos run their microservices with consistent configurations at scale, along with Aurora for long-running services and cron jobs.

See more

related Nomad posts

Robert Zuber

Our backend consists of two major pools of machines. One pool hosts the systems that run our site, manage jobs, and send notifications. These services are deployed within Docker containers orchestrated in Kubernetes. Due to Kubernetes’ ecosystem and toolchain, it was an obvious choice for our fairly statically-defined processes: the rate of change of job types or how many we may need in our internal stack is relatively low.

The other pool of machines is for running our users’ jobs. Because we cannot dynamically predict demand, what types of jobs our users need to have run, nor the resources required for each of those jobs, we found that Nomad excelled over Kubernetes in this area.

We’re also using Helm to make it easier to deploy new services into Kubernetes. We create a chart (i.e. package) for each service. This lets us easily roll back new software and gives us an audit trail of what was installed or upgraded.

See more
DC/OS logo

DC/OS

107
148
12
The Datacenter Operating System. The easiest way to run microservices, big data, and containers in production.
107
148
+ 1
12

related DC/OS posts

Mesosphere logo

Mesosphere

79
97
6
Combine your datacenter servers and cloud instances into one shared pool
79
97
+ 1
6
PROS OF MESOSPHERE
CONS OF MESOSPHERE
    No cons available

    related Mesosphere posts

    kops logo

    kops

    72
    53
    0
    Production Grade K8s Installation, Upgrades, and Management
    72
    53
    + 1
    0
    PROS OF KOPS
      No pros available
      CONS OF KOPS
        No cons available

        related kops posts

        Apache Aurora logo

        Apache Aurora

        56
        67
        0
        An Apcahe Mesos framework for scheduling jobs, originally developed by Twitter
        56
        67
        + 1
        0
        PROS OF APACHE AURORA
          No pros available
          CONS OF APACHE AURORA
            No cons available

            related Apache Aurora posts

            Docker containers on Mesos run their microservices with consistent configurations at scale, along with Aurora for long-running services and cron jobs.

            See more
            Gardener logo

            Gardener

            10
            29
            1
            Manage Kubernetes clusters across multiple cloud providers
            10
            29
            + 1
            1
            CONS OF GARDENER
              No cons available

              related Gardener posts