Alternatives to AWS Step Functions logo

Alternatives to AWS Step Functions

AWS Lambda, Airflow, AWS Batch, Batch, and AWS Data Pipeline are the most popular alternatives and competitors to AWS Step Functions.
100
162
+ 1
11

What is AWS Step Functions and what are its top alternatives?

AWS Step Functions makes it easy to coordinate the components of distributed applications and microservices using visual workflows. Building applications from individual components that each perform a discrete function lets you scale and change applications quickly.
AWS Step Functions is a tool in the Cloud Task Management category of a tech stack.

AWS Step Functions alternatives & related posts

AWS Lambda logo

AWS Lambda

9.1K
7.2K
395
9.1K
7.2K
+ 1
395
Automatically run code in response to modifications to objects in Amazon S3 buckets, messages in Kinesis streams, or...
AWS Lambda logo
AWS Lambda
VS
AWS Step Functions logo
AWS Step Functions

related AWS Lambda posts

Jeyabalaji Subramanian
Jeyabalaji Subramanian
CTO at FundsCorner | 25 upvotes 1.1M views

Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

Based on the above criteria, we selected the following tools to perform the end to end data replication:

We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!

See more
Tim Nolet
Tim Nolet
Founder, Engineer & Dishwasher at Checkly | 20 upvotes 1.4M views

Heroku Docker GitHub Node.js hapi Vue.js AWS Lambda Amazon S3 PostgreSQL Knex.js Checkly is a fairly young company and we're still working hard to find the correct mix of product features, price and audience.

We are focussed on tech B2B, but I always wanted to serve solo developers too. So I decided to make a $7 plan.

Why $7? Simply put, it seems to be a sweet spot for tech companies: Heroku, Docker, Github, Appoptics (Librato) all offer $7 plans. They must have done a ton of research into this, so why not piggy back that and try it out.

Enough biz talk, onto tech. The challenges were:

  • Slice of a portion of the functionality so a $7 plan is still profitable. We call this the "plan limits"
  • Update API and back end services to handle and enforce plan limits.
  • Update the UI to kindly state plan limits are in effect on some part of the UI.
  • Update the pricing page to reflect all changes.
  • Keep the actual processing backend, storage and API's as untouched as possible.

In essence, we went from strictly volume based pricing to value based pricing. Here come the technical steps & decisions we made to get there.

  1. We updated our PostgreSQL schema so plans now have an array of "features". These are string constants that represent feature toggles.
  2. The Vue.js frontend reads these from the vuex store on login.
  3. Based on these values, the UI has simple v-if statements to either just show the feature or show a friendly "please upgrade" button.
  4. The hapi API has a hook on each relevant API endpoint that checks whether a user's plan has the feature enabled, or not.

Side note: We offer 10 SMS messages per month on the developer plan. However, we were not actually counting how many people were sending. We had to update our alerting daemon (that runs on Heroku and triggers SMS messages via AWS SNS) to actually bump a counter.

What we build is basically feature-toggling based on plan features. It is very extensible for future additions. Our scheduling and storage backend that actually runs users' monitoring requests (AWS Lambda) and stores the results (S3 and Postgres) has no knowledge of all of this and remained unchanged.

Hope this helps anyone building out their SaaS and is in a similar situation.

See more
Airflow logo

Airflow

684
1K
57
684
1K
+ 1
57
A platform to programmaticaly author, schedule and monitor data pipelines, by Airbnb
Airflow logo
Airflow
VS
AWS Step Functions logo
AWS Step Functions

related Airflow posts

StackShare Editors
StackShare Editors

Since the beginning, Cal Henderson has been the CTO of Slack. Earlier this year, he commented on a Quora question summarizing their current stack.

Apps
  • Web: a mix of JavaScript/ES6 and React.
  • Desktop: And Electron to ship it as a desktop application.
  • Android: a mix of Java and Kotlin.
  • iOS: written in a mix of Objective C and Swift.
Backend
  • The core application and the API written in PHP/Hack that runs on HHVM.
  • The data is stored in MySQL using Vitess.
  • Caching is done using Memcached and MCRouter.
  • The search service takes help from SolrCloud, with various Java services.
  • The messaging system uses WebSockets with many services in Java and Go.
  • Load balancing is done using HAproxy with Consul for configuration.
  • Most services talk to each other over gRPC,
  • Some Thrift and JSON-over-HTTP
  • Voice and video calling service was built in Elixir.
Data warehouse
  • Built using open source tools including Presto, Spark, Airflow, Hadoop and Kafka.
Etc
See more
AWS Batch logo

AWS Batch

27
92
4
27
92
+ 1
4
Fully Managed Batch Processing at Any Scale
AWS Batch logo
AWS Batch
VS
AWS Step Functions logo
AWS Step Functions
Batch logo

Batch

24
14
1
24
14
+ 1
1
Free retention toolkit for indie developers & startups - push notifications, user analytics, reward engine, and native ads
Batch logo
Batch
VS
AWS Step Functions logo
AWS Step Functions
AWS Data Pipeline logo

AWS Data Pipeline

68
228
1
68
228
+ 1
1
Process and move data between different AWS compute and storage services
AWS Data Pipeline logo
AWS Data Pipeline
VS
AWS Step Functions logo
AWS Step Functions
Amazon SWF logo

Amazon SWF

33
41
0
33
41
+ 1
0
Automate the coordination, auditing, and scaling of applications across multiple machines
    Be the first to leave a pro
    Amazon SWF logo
    Amazon SWF
    VS
    AWS Step Functions logo
    AWS Step Functions
    Google Keep logo

    Google Keep

    25
    18
    0
    25
    18
    + 1
    0
    Capture what鈥檚 important and get more done
      Be the first to leave a pro
      Google Keep logo
      Google Keep
      VS
      AWS Step Functions logo
      AWS Step Functions
      Workfront logo

      Workfront

      7
      4
      0
      7
      4
      + 1
      0
      A platform for enterprise work management
        Be the first to leave a pro
        Workfront logo
        Workfront
        VS
        AWS Step Functions logo
        AWS Step Functions