Swiftype Explains Their Cloud Stack

At Swiftype our mission is to make it easy for anyone to add great search functionality to their website or mobile application. We believe that search is one of the most critical interactions between a product and a user, since it is one of the most explicit expressions of intent your users can reveal to you. My co-founder Quin and I learned this firsthand as engineers at Scribd.com, where we built one of the largest full-text search engines on the web. At Scribd, search was an incredibly important problem. At a product level, millions of users searched the site every day, so search satisfaction was a key discovery and retention path. At a technical level, the wide variety and sheer mass of content uploaded presented unique challenges that required significant search expertise to accommodate.

As with many startups, Scribd first attempted to buy a hosted search solution that would allow us to move faster, but after evaluating the market we couldn’t find anything that met our needs. So Scribd developed its own solution internally based on some great open source tools and learned very quickly how difficult and expensive it is to deliver a truly great site search experience. Quin and I realized the trouble Scribd had buying a solution was a great opportunity for a new company, so we started Swiftype. <!--more--> Developers are typically the earliest adopters of cloud services, and ours is no different. Developers love Swiftype because it allows them to quickly develop a better, higher quality search engine than what they could reasonably do on their own. And because it’s cloud based, we can scale the solution automatically as your product grows. For us, performance and stability are features in their own right, so we’re regularly iterating on and improving our own stack. Our servers are hosted on dedicated hardware in multiple datacenters at SoftLayer. For an application with high I/O, CPU, and memory requirements, there really isn’t any substitute for dedicated hardware. EC2 can be a great fit for a certain profile of web applications, but it really didn’t perform at the level we needed for the service we strive to deliver, and that vastly outweighed the convenience you get from EC2. There was a similar impetus for multi-datacenter support — for an application like search, speed is critical, and latency is a huge part of that for most users. Being able to geographically distribute our search servers means that we can deliver results more quickly to users around the world.

Implementing a better product with less hassle is really only half the advantage of using a service like ours. The other half — which doesn't seem to get as much marketing play — is that by leveraging the product of a company dedicated to a single, specific technology, you realize the gains of having a full-time team of domain experts dedicated to improving your search feature, without assuming any of the cost. At Swiftype we spend all of our time thinking about, developing, and iterating on search, and every time we ship an improvement, all of our customers reap the benefits instantly. Our experience has shown that at most companies it can be a full-time job just maintaining an internal search system, much less improving it over time. When search isn't a core competency of your company, we believe you're better off letting us take care of the details.

And of course the same philosophy applies to our company as well, which is why we leverage so many existing cloud-based services in our daily operations. Anywhere that we can save time and resources using a product that another company focuses their full effort on delivering is a win for us, because it allows us to spend our resources on what we do best — building great search software. Here's what we use at Swiftype:

• SoftLayer "All our hardware is hosted at SoftLayer and we’re very happy. Great performance, reliable support, and good prices. We switched here from EC2 and never looked back."

• imgix "Image resizing and caching via simple URL-based API. We use this to format thumbnails in search results of pages we crawl. Never installed ImageMagick on a single Swiftype server."

• Sentry "Pretty and fast exception tracking. Fast is important — when your site is throwing exceptions there is nothing worse than waiting for a 10 second refresh just to see the latest exceptions in the dashboard. I could complain for days about our former tracker, but instead I’ll just say Sentry does it better."

• New Relic "Expensive but worth it. Monitors almost all of our core performance metrics, and we show our dashboard on a big LCD in our office 24x7"

• Fastly "CDN with a sane API, and competitive prices. Cool dudes, too."

• CircleCI "Great tool for running tests continuously. Only complaint is that it tells the whole HipChat when I break the build :(

• Pingdom "Every app has it and we’re no different."

• PagerDuty "Very reliable and flexible tool for handling on-call duty."

• HipChat "Thankfully they just released a native Mac app. Love the integrations with other services (Github, SupportBee, etc.)"

• GitHub "The best way to write code as a team. It’s fully integrated into our workflow, and especially helpful for developing our public-facing developer tools."

• MailChimp "Great for managing weekly or monthly newsletters, ensuring browser compatibility, and learning about how effective your campaigns are over time"

• SupportBee "Awesome, up-and-coming support desk software that fits our workflow very well. You still primarily interact with tickets via email, but you can use their web interface for a lot as well. Also highly recommended."

• Dropbox "Simple filesharing is good."

• Google Apps "If only we could get our Lawyers to use it too."

• Olark "Great for communicating with customers in real-time, also lets us know when important clients are on the site"

• Amazon S3 "S3 has been fantastic"

• Asana "Awesome tool for task management and productivity"

• Mixpanel "We use their funnel analysis for all the important flows on our site"

• Stripe "A great way to get started with payments online, and rock-solid developer tools that you can trust"

• Mailgun "Awesome, dead-simple tool for sending emails from your app. The service just works. Highly recommended."

• Google Analytics "Pretty much the standard for tracking traffic, inbound referrals, etc."

You can follow Swiftype on Twitter @swiftype. To see what other startups and companies are using, check out [Cloud Stacks].