Alternatives to Apache Dubbo logo

Alternatives to Apache Dubbo

Spring Cloud, gRPC, Apache Thrift, Node.js, and Kubernetes are the most popular alternatives and competitors to Apache Dubbo.
23
43
+ 1
0

What is Apache Dubbo and what are its top alternatives?

It is a high-performance, light weight, java based RPC framework. Dubbo offers three key functionalities, which include interface based remote call, fault tolerance & load balancing, and automatic service registration & discovery.
Apache Dubbo is a tool in the Remote Procedure Call (RPC) category of a tech stack.
Apache Dubbo is an open source tool with 36.5K GitHub stars and 24.3K GitHub forks. Here’s a link to Apache Dubbo's open source repository on GitHub

Top Alternatives to Apache Dubbo

  • Spring Cloud

    Spring Cloud

    It provides tools for developers to quickly build some of the common patterns in distributed systems. ...

  • gRPC

    gRPC

    gRPC is a modern open source high performance RPC framework that can run in any environment. It can efficiently connect services in and across data centers with pluggable support for load balancing, tracing, health checking... ...

  • Apache Thrift

    Apache Thrift

    The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages. ...

  • Node.js

    Node.js

    Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices. ...

  • Kubernetes

    Kubernetes

    Kubernetes is an open source orchestration system for Docker containers. It handles scheduling onto nodes in a compute cluster and actively manages workloads to ensure that their state matches the users declared intentions. ...

  • Django

    Django

    Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. ...

  • ASP.NET

    ASP.NET

    .NET is a developer platform made up of tools, programming languages, and libraries for building many different types of applications. ...

  • Laravel

    Laravel

    It is a web application framework with expressive, elegant syntax. It attempts to take the pain out of development by easing common tasks used in the majority of web projects, such as authentication, routing, sessions, and caching. ...

Apache Dubbo alternatives & related posts

Spring Cloud logo

Spring Cloud

708
627
0
Spring helps development teams everywhere build simple, portable,fast and flexible JVM-based systems and applications.
708
627
+ 1
0
PROS OF SPRING CLOUD
    Be the first to leave a pro
    CONS OF SPRING CLOUD
      Be the first to leave a con

      related Spring Cloud posts

      Spring-Boot Spring Cloud Elasticsearch MySQL Redis RabbitMQ Kafka MongoDB GitHub Linux IntelliJ IDEA

      See more
      gRPC logo

      gRPC

      878
      952
      46
      A high performance, open-source universal RPC framework
      878
      952
      + 1
      46
      PROS OF GRPC
      • 19
        Higth performance
      • 10
        The future of API
      • 10
        Easy setup
      • 4
        Contract-based
      • 3
        Polyglot
      CONS OF GRPC
        Be the first to leave a con

        related gRPC posts

        Shared insights
        on
        gRPCgRPCSignalRSignalR.NET.NET

        We need to interact from several different Web applications (remote) to a client-side application (.exe in .NET Framework, Windows.Console under our controlled environment). From the web applications, we need to send and receive data and invoke methods to client-side .exe on javascript events like users onclick. SignalR is one of the .Net alternatives to do that, but it adds overhead for what we need. Is it better to add SignalR at both client-side application and remote web application, or use gRPC as it sounds lightest and is multilingual?

        SignalR or gRPC are always sending and receiving data on the client-side (from browser to .exe and back to browser). And web application is used for graphical visualization of data to the user. There is no need for local .exe to send or interact with remote web API. Which architecture or framework do you suggest to use in this case?

        See more
        Shared insights
        on
        KafkaKafkagRPCgRPC
        at

        By mid-2015, Uber’s rider growth coupled with its cadence of releasing new services, like Eats and Freight, was pressuring the infrastructure. To allow the decoupling of consumption from production, and to add an abstraction layer between users, developers, and infrastructure, Uber built Catalyst, a serverless internal service mesh.

        Uber decided to build their own severless solution, rather that using something like AWS Lambda, speed for its global production environments as well as introspectability.

        See more
        Apache Thrift logo

        Apache Thrift

        109
        190
        0
        Software framework for scalable cross-language services development
        109
        190
        + 1
        0
        PROS OF APACHE THRIFT
          Be the first to leave a pro
          CONS OF APACHE THRIFT
            Be the first to leave a con

            related Apache Thrift posts

            Since the beginning, Cal Henderson has been the CTO of Slack. Earlier this year, he commented on a Quora question summarizing their current stack.

            Apps
            • Web: a mix of JavaScript/ES6 and React.
            • Desktop: And Electron to ship it as a desktop application.
            • Android: a mix of Java and Kotlin.
            • iOS: written in a mix of Objective C and Swift.
            Backend
            • The core application and the API written in PHP/Hack that runs on HHVM.
            • The data is stored in MySQL using Vitess.
            • Caching is done using Memcached and MCRouter.
            • The search service takes help from SolrCloud, with various Java services.
            • The messaging system uses WebSockets with many services in Java and Go.
            • Load balancing is done using HAproxy with Consul for configuration.
            • Most services talk to each other over gRPC,
            • Some Thrift and JSON-over-HTTP
            • Voice and video calling service was built in Elixir.
            Data warehouse
            • Built using open source tools including Presto, Spark, Airflow, Hadoop and Kafka.
            Etc
            See more
            Node.js logo

            Node.js

            125K
            103.6K
            8.4K
            A platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications
            125K
            103.6K
            + 1
            8.4K
            PROS OF NODE.JS
            • 1.4K
              Npm
            • 1.3K
              Javascript
            • 1.1K
              Great libraries
            • 1K
              High-performance
            • 795
              Open source
            • 484
              Great for apis
            • 474
              Asynchronous
            • 420
              Great community
            • 390
              Great for realtime apps
            • 295
              Great for command line utilities
            • 81
              Node Modules
            • 80
              Websockets
            • 67
              Uber Simple
            • 57
              Great modularity
            • 56
              Allows us to reuse code in the frontend
            • 40
              Easy to start
            • 35
              Great for Data Streaming
            • 31
              Realtime
            • 26
              Awesome
            • 24
              Non blocking IO
            • 17
              Can be used as a proxy
            • 16
              High performance, open source, scalable
            • 15
              Non-blocking and modular
            • 14
              Easy and Fun
            • 12
              Easy and powerful
            • 12
              Same lang as AngularJS
            • 11
              Future of BackEnd
            • 10
              Fast
            • 9
              Fullstack
            • 9
              Scalability
            • 9
              Cross platform
            • 8
              Simple
            • 7
              Mean Stack
            • 6
              Great for webapps
            • 6
              Easy concurrency
            • 5
              Friendly
            • 5
              Fast, simple code and async
            • 5
              React
            • 4
              Great speed
            • 4
              Fast development
            • 4
              Its amazingly fast and scalable
            • 4
              Control everything
            • 4
              Easy to use and fast and goes well with JSONdb's
            • 4
              Typescript
            • 4
              Scalable
            • 3
              It's fast
            • 3
              Easy to use
            • 3
              Isomorphic coolness
            • 2
              Sooper easy for the Backend connectivity
            • 2
              Easy to learn
            • 2
              TypeScript Support
            • 2
              Scales, fast, simple, great community, npm, express
            • 2
              One language, end-to-end
            • 2
              Javascript2
            • 2
              Not Python
            • 2
              Performant and fast prototyping
            • 2
              Blazing fast
            • 2
              Great community
            • 2
              Less boilerplate code
            • 2
              Easy
            • 1
              Lovely
            • 1
              Event Driven
            CONS OF NODE.JS
            • 46
              Bound to a single CPU
            • 42
              New framework every day
            • 35
              Lots of terrible examples on the internet
            • 29
              Asynchronous programming is the worst
            • 23
              Callback
            • 16
              Javascript
            • 11
              Dependency based on GitHub
            • 10
              Dependency hell
            • 10
              Low computational power
            • 7
              Can block whole server easily
            • 6
              Very very Slow
            • 6
              Callback functions may not fire on expected sequence
            • 3
              Unneeded over complication
            • 3
              Unstable
            • 3
              Breaking updates
            • 1
              No standard approach

            related Node.js posts

            Nick Rockwell
            SVP, Engineering at Fastly · | 44 upvotes · 1.7M views

            When I joined NYT there was already broad dissatisfaction with the LAMP (Linux Apache HTTP Server MySQL PHP) Stack and the front end framework, in particular. So, I wasn't passing judgment on it. I mean, LAMP's fine, you can do good work in LAMP. It's a little dated at this point, but it's not ... I didn't want to rip it out for its own sake, but everyone else was like, "We don't like this, it's really inflexible." And I remember from being outside the company when that was called MIT FIVE when it had launched. And been observing it from the outside, and I was like, you guys took so long to do that and you did it so carefully, and yet you're not happy with your decisions. Why is that? That was more the impetus. If we're going to do this again, how are we going to do it in a way that we're gonna get a better result?

            So we're moving quickly away from LAMP, I would say. So, right now, the new front end is React based and using Apollo. And we've been in a long, protracted, gradual rollout of the core experiences.

            React is now talking to GraphQL as a primary API. There's a Node.js back end, to the front end, which is mainly for server-side rendering, as well.

            Behind there, the main repository for the GraphQL server is a big table repository, that we call Bodega because it's a convenience store. And that reads off of a Kafka pipeline.

            See more
            Conor Myhrvold
            Tech Brand Mgr, Office of CTO at Uber · | 39 upvotes · 4.2M views

            How Uber developed the open source, end-to-end distributed tracing Jaeger , now a CNCF project:

            Distributed tracing is quickly becoming a must-have component in the tools that organizations use to monitor their complex, microservice-based architectures. At Uber, our open source distributed tracing system Jaeger saw large-scale internal adoption throughout 2016, integrated into hundreds of microservices and now recording thousands of traces every second.

            Here is the story of how we got here, from investigating off-the-shelf solutions like Zipkin, to why we switched from pull to push architecture, and how distributed tracing will continue to evolve:

            https://eng.uber.com/distributed-tracing/

            (GitHub Pages : https://www.jaegertracing.io/, GitHub: https://github.com/jaegertracing/jaeger)

            Bindings/Operator: Python Java Node.js Go C++ Kubernetes JavaScript OpenShift C# Apache Spark

            See more
            Kubernetes logo

            Kubernetes

            38.9K
            33K
            628
            Manage a cluster of Linux containers as a single system to accelerate Dev and simplify Ops
            38.9K
            33K
            + 1
            628
            PROS OF KUBERNETES
            • 159
              Leading docker container management solution
            • 124
              Simple and powerful
            • 101
              Open source
            • 75
              Backed by google
            • 56
              The right abstractions
            • 24
              Scale services
            • 18
              Replication controller
            • 9
              Permission managment
            • 7
              Simple
            • 7
              Supports autoscaling
            • 6
              Cheap
            • 4
              Self-healing
            • 4
              Reliable
            • 4
              No cloud platform lock-in
            • 3
              Open, powerful, stable
            • 3
              Scalable
            • 3
              Quick cloud setup
            • 3
              Promotes modern/good infrascture practice
            • 2
              Backed by Red Hat
            • 2
              Runs on azure
            • 2
              Cloud Agnostic
            • 2
              Custom and extensibility
            • 2
              Captain of Container Ship
            • 2
              A self healing environment with rich metadata
            • 1
              Golang
            • 1
              Easy setup
            • 1
              Everything of CaaS
            • 1
              Sfg
            • 1
              Expandable
            • 1
              Gke
            CONS OF KUBERNETES
            • 13
              Poor workflow for development
            • 11
              Steep learning curve
            • 5
              Orchestrates only infrastructure
            • 2
              High resource requirements for on-prem clusters

            related Kubernetes posts

            Conor Myhrvold
            Tech Brand Mgr, Office of CTO at Uber · | 39 upvotes · 4.2M views

            How Uber developed the open source, end-to-end distributed tracing Jaeger , now a CNCF project:

            Distributed tracing is quickly becoming a must-have component in the tools that organizations use to monitor their complex, microservice-based architectures. At Uber, our open source distributed tracing system Jaeger saw large-scale internal adoption throughout 2016, integrated into hundreds of microservices and now recording thousands of traces every second.

            Here is the story of how we got here, from investigating off-the-shelf solutions like Zipkin, to why we switched from pull to push architecture, and how distributed tracing will continue to evolve:

            https://eng.uber.com/distributed-tracing/

            (GitHub Pages : https://www.jaegertracing.io/, GitHub: https://github.com/jaegertracing/jaeger)

            Bindings/Operator: Python Java Node.js Go C++ Kubernetes JavaScript OpenShift C# Apache Spark

            See more
            Yshay Yaacobi

            Our first experience with .NET core was when we developed our OSS feature management platform - Tweek (https://github.com/soluto/tweek). We wanted to create a solution that is able to run anywhere (super important for OSS), has excellent performance characteristics and can fit in a multi-container architecture. We decided to implement our rule engine processor in F# , our main service was implemented in C# and other components were built using JavaScript / TypeScript and Go.

            Visual Studio Code worked really well for us as well, it worked well with all our polyglot services and the .Net core integration had great cross-platform developer experience (to be fair, F# was a bit trickier) - actually, each of our team members used a different OS (Ubuntu, macos, windows). Our production deployment ran for a time on Docker Swarm until we've decided to adopt Kubernetes with almost seamless migration process.

            After our positive experience of running .Net core workloads in containers and developing Tweek's .Net services on non-windows machines, C# had gained back some of its popularity (originally lost to Node.js), and other teams have been using it for developing microservices, k8s sidecars (like https://github.com/Soluto/airbag), cli tools, serverless functions and other projects...

            See more
            Django logo

            Django

            26.6K
            23.5K
            3.8K
            The Web framework for perfectionists with deadlines
            26.6K
            23.5K
            + 1
            3.8K
            PROS OF DJANGO
            • 634
              Rapid development
            • 468
              Open source
            • 401
              Great community
            • 353
              Easy to learn
            • 263
              Mvc
            • 215
              Beautiful code
            • 210
              Elegant
            • 193
              Free
            • 191
              Great packages
            • 178
              Great libraries
            • 68
              Restful
            • 65
              Comes with auth and crud admin panel
            • 65
              Powerful
            • 60
              Great documentation
            • 58
              Great for web
            • 44
              Python
            • 37
              Great orm
            • 34
              Great for api
            • 27
              All included
            • 22
              Web Apps
            • 21
              Fast
            • 18
              Used by top startups
            • 16
              Clean
            • 15
              Easy setup
            • 15
              Sexy
            • 12
              Convention over configuration
            • 10
              ORM
            • 9
              The Django community
            • 9
              Allows for very rapid development with great libraries
            • 6
              Great MVC and templating engine
            • 6
              King of backend world
            • 6
              Its elegant and practical
            • 5
              Mvt
            • 5
              Batteries included
            • 5
              Full stack
            • 5
              Fast prototyping
            • 5
              Easy Structure , useful inbuilt library
            • 5
              Easy to develop end to end AI Models
            • 5
              Have not found anything that it can't do
            • 4
              Very quick to get something up and running
            • 4
              Easy to use
            • 4
              Easy
            • 4
              Cross-Platform
            • 3
              Map
            • 3
              Great peformance
            • 3
              Scaffold
            • 3
              Just the right level of abstraction
            • 3
              Modular
            • 3
              Full-Text Search
            • 3
              Zero code burden to change databases
            • 3
              Python community
            • 3
              Many libraries
            • 2
              Easy to change database manager
            • 1
              Node js
            CONS OF DJANGO
            • 25
              Underpowered templating
            • 19
              Underpowered ORM
            • 19
              Autoreload restarts whole server
            • 15
              URL dispatcher ignores HTTP method
            • 10
              Internal subcomponents coupling
            • 7
              Admin
            • 7
              Not nodejs
            • 6
              Configuration hell
            • 4
              Not as clean and nice documentation like Laravel
            • 3
              Python
            • 3
              Not typed
            • 3
              Bloated admin panel included
            • 2
              Overwhelming folder structure
            • 1
              InEffective Multithreading

            related Django posts

            Dmitry Mukhin

            Simple controls over complex technologies, as we put it, wouldn't be possible without neat UIs for our user areas including start page, dashboard, settings, and docs.

            Initially, there was Django. Back in 2011, considering our Python-centric approach, that was the best choice. Later, we realized we needed to iterate on our website more quickly. And this led us to detaching Django from our front end. That was when we decided to build an SPA.

            For building user interfaces, we're currently using React as it provided the fastest rendering back when we were building our toolkit. It’s worth mentioning Uploadcare is not a front-end-focused SPA: we aren’t running at high levels of complexity. If it were, we’d go with Ember.js.

            However, there's a chance we will shift to the faster Preact, with its motto of using as little code as possible, and because it makes more use of browser APIs. One of our future tasks for our front end is to configure our Webpack bundler to split up the code for different site sections. For styles, we use PostCSS along with its plugins such as cssnano which minifies all the code.

            All that allows us to provide a great user experience and quickly implement changes where they are needed with as little code as possible.

            See more

            Hey, so I developed a basic application with Python. But to use it, you need a python interpreter. I want to add a GUI to make it more appealing. What should I choose to develop a GUI? I have very basic skills in front end development (CSS, JavaScript). I am fluent in python. I'm looking for a tool that is easy to use and doesn't require too much code knowledge. I have recently tried out Flask, but it is kinda complicated. Should I stick with it, move to Django, or is there another nice framework to use?

            See more
            ASP.NET logo

            ASP.NET

            21.4K
            6.7K
            14
            An open source web framework for building modern web apps and services with .NET
            21.4K
            6.7K
            + 1
            14
            PROS OF ASP.NET
            • 11
              Great mvc
            • 3
              Easy to learn
            CONS OF ASP.NET
            • 1
              Not highly flexible for advance Developers
            • 1
              Entity framework is very slow

            related ASP.NET posts

            Greg Neumann

            Finding the most effective dev stack for a solo developer. Over the past year, I've been looking at many tech stacks that would be 'best' for me, as a solo, indie, developer to deliver a desktop app (Windows & Mac) plus mobile - iOS mainly. Initially, Xamarin started to stand-out. Using .NET Core as the run-time, Xamarin as the native API provider and Xamarin Forms for the UI seemed to solve all issues. But, the cracks soon started to appear. Xamarin Forms is mobile only; the Windows incarnation is different. There is no Mac UI solution (you have to code it natively in Mac OS Storyboard. I was also worried how Xamarin Forms , if I was to use it, was going to cope, in future, with Apple's new SwiftUI and Google's new Fuchsia.

            This plethora of techs for the UI-layer made me reach for the safer waters of using Web-techs for the UI. Lovely! Consistency everywhere (well, mostly). But that consistency evaporates when platform issues are addressed. There are so many web frameworks!

            But, I made a simple decision. It's just me...I am clever, but there is no army of coders here. And I have big plans for a business app. How could just 1 developer go-on to deploy a decent app to Windows, iPhone, iPad & Mac OS? I remembered earlier days when I've used Microsoft's ASP.NET to scaffold - generate - loads of Code for a web-app that I needed for several charities that I worked with. What 'generators' exist that do a lot of the platform-specific rubbish, allow the necessary customisation of such platform integration and provide a decent UI?

            I've placed my colours to the Quasar Framework mast. Oh dear, that means Electron desktop apps doesn't it? Well, Ive had enough of loads of Developers saying that "the menus won't look native" or "it uses too much RAM" and so on. I've been using non-native UI-wrapped apps for ages - the date picker in Outlook on iOS is way better than the native date-picker and I'd been using it for years without getting hot under the collar about it. Developers do get so hung-up on things that busy Users hardly notice; don't you think?. As to the RAM usage issue; that's a bit true. But Users only really notice when an app uses so much RAM that the machine starts to page-out. Electron contributes towards that horizon but does not cause it. My Users will be business-users after all. Somewhat decent machines.

            Looking forward to all that lovely Vue.js around my TypeScript and all those really, really, b e a u t I f u l UI controls of Quasar Framework . Still not sure that 1 dev can deliver all that... but I'm up for trying...

            See more

            Hi. We are planning to develop web, desktop, and mobile app for procurement, logistics, and contracts. Procure to Pay and Source to pay, spend management, supplier management, catalog management. ( similar to SAP Ariba, gap.com, coupa.com, ivalua.com vroozi.com, procurify.com

            We got stuck when deciding which technology stack is good for the future. We look forward to your kind guidance that will help us.

            We want to integrate with multiple databases with seamless bidirectional integration. What APIs and middleware available are best to achieve this? SAP HANA, Oracle, MySQL, MongoDB...

            ASP.NET / Node.js / Laravel. ......?

            Please guide us

            See more
            Laravel logo

            Laravel

            20.4K
            16.2K
            3.5K
            A PHP Framework For Web Artisans
            20.4K
            16.2K
            + 1
            3.5K
            PROS OF LARAVEL
            • 514
              Clean architecture
            • 374
              Growing community
            • 348
              Composer friendly
            • 323
              Open source
            • 303
              The only framework to consider for php
            • 202
              Mvc
            • 195
              Quickly develop
            • 159
              Dependency injection
            • 147
              Application architecture
            • 134
              Embraces good community packages
            • 62
              Write less, do more
            • 55
              Restful routing
            • 52
              Orm (eloquent)
            • 47
              Artisan scaffolding and migrations
            • 47
              Database migrations & seeds
            • 36
              Awesome
            • 34
              Great documentation
            • 26
              Awsome, Powerfull, Fast and Rapid
            • 25
              Promotes elegant coding
            • 25
              Build Apps faster, easier and better
            • 22
              Easy to learn, scalability
            • 22
              JSON friendly
            • 21
              Eloquent ORM
            • 21
              Modern PHP
            • 21
              Most easy for me
            • 20
              Test-Driven
            • 19
              Blade Template
            • 18
              Beautiful
            • 13
              Security
            • 12
              Based on SOLID
            • 11
              Simple
            • 11
              Clean Documentation
            • 11
              Cool
            • 10
              Easy to attach Middleware
            • 10
              Convention over Configuration
            • 9
              Easy Request Validatin
            • 8
              Easy to use
            • 8
              Its just wow
            • 8
              Simpler
            • 8
              Get going quickly straight out of the box. BYOKDM
            • 8
              Laravel + Cassandra = Killer Framework
            • 8
              Fast
            • 7
              Less dependencies
            • 7
              Friendly API
            • 7
              Super easy and powerful
            • 7
              Simplistic , easy and faster
            • 6
              Great customer support
            • 6
              Its beautiful to code in
            • 5
              The only "cons" is wrong! No static method just Facades
            • 5
              Fast and Clarify framework
            • 5
              Active Record
            • 5
              Php7
            • 5
              Speed
            • 5
              Easy
            • 4
              Composer
            • 4
              Laravel Mix
            • 4
              Minimum system requirements
            • 4
              Easy views handling and great ORM
            • 4
              Eloquent
            • 4
              Laragon
            • 3
              Laravel Nova
            • 3
              Laravel casher
            • 3
              Laravel Passport
            • 3
              Ease of use
            • 3
              Cashier with Braintree and Stripe
            • 3
              Laravel Forge and Envoy
            • 3
              Laravel Horizon and Telescope
            • 3
              Laravel Spark
            • 3
              Intuitive usage
            • 2
              Heart touch
            • 2
              Like heart beat
            • 2
              Rapid development
            • 2
              Laravel love live long
            • 2
              Touch heart artisan
            • 2
              Scout
            CONS OF LARAVEL
            • 42
              PHP
            • 28
              Too many dependency
            • 20
              Slower than the other two
            • 16
              A lot of static method calls for convenience
            • 13
              Too many include
            • 10
              Heavy
            • 7
              Bloated
            • 6
              Laravel
            • 5
              Confusing
            • 4
              Does not work well for file uploads in Shared Hosting
            • 4
              Too underrated
            • 2
              Not fast with MongoDB
            • 1
              Difficult to learn
            • 1
              Not using SOLID principles

            related Laravel posts

            I need to build a web application plus android and IOS apps for an enterprise, like an e-commerce portal. It will have intensive use of MySQL to display thousands (40-50k) of live product information in an interactive table (searchable, filterable), live delivery tracking. It has to be secure, as it will handle information on customers, sales, inventory. Here is the technology stack: Backend: Laravel 7 Frondend: Vue.js, React or AngularJS?

            Need help deciding technology stack. Thanks.

            See more
            Antonio Sanchez

            Back at the start of 2017, we decided to create a web-based tool for the SEO OnPage analysis of our clients' websites. We had over 2.000 websites to analyze, so we had to perform thousands of requests to get every single page from those websites, process the information and save the big amounts of data somewhere.

            Very soon we realized that the initial chosen script language and database, PHP, Laravel and MySQL, was not going to be able to cope efficiently with such a task.

            By that time, we were doing some experiments for other projects with a language we had recently get to know, Go , so we decided to get a try and code the crawler using it. It was fantastic, we could process much more data with way less CPU power and in less time. By using the concurrency abilites that the language has to offers, we could also do more Http requests in less time.

            Unfortunately, I have no comparison numbers to show about the performance differences between Go and PHP since the difference was so clear from the beginning and that we didn't feel the need to do further comparison tests nor document it. We just switched fully to Go.

            There was still a problem: despite the big amount of Data we were generating, MySQL was performing very well, but as we were adding more and more features to the software and with those features more and more different type of data to save, it was a nightmare for the database architects to structure everything correctly on the database, so it was clear what we had to do next: switch to a NoSQL database. So we switched to MongoDB, and it was also fantastic: we were expending almost zero time in thinking how to structure the Database and the performance also seemed to be better, but again, I have no comparison numbers to show due to the lack of time.

            We also decided to switch the website from PHP and Laravel to JavaScript and Node.js and ExpressJS since working with the JSON Data that we were saving now in the Database would be easier.

            As of now, we don't only use the tool intern but we also opened it for everyone to use for free: https://tool-seo.com

            See more