Zulip

We use DigitalOcean mainly to provide remote development environments for Zulip contributors in situations where developing locally using our Vagrant setup isn't practical. There's a range of reasons:

• Situations where one needs a public IP address and SSL certificate (e.g. Facebook's OAuth system require that even for testing)
• Giving a contributor a development environment when their computer doesn't have the few GB of free RAM needed to run one locally
• Developer sprints, where our snapshot-based system can provision a working development environment for a potential new contributor in under a minute. This use case is particularly great because a machine that one only needs for 3 days is essentially free with Digital Ocean's pricing.
• A backup development environment when someone's laptop is being repaired.

One could do all of this with many hosting providers, but we've found it particularly convenient to use Digital Ocean for these applications.

We use VirtualBox primarily as a Vagrant provider for macOS for the Zulip development environment. It's totally reasonable software for providing a convenient virtual machine setup on macOS (and for debugging when things go wrong, which is mostly how we use it since the Vagrant provider for macOS just works).

We use Vagrant because it is the best toolchain for having a standardized development environment that is readily provisoned with just a single command on macOS, Linux, and Windows.

There's a lot of things that could be better; the thing I dislike the most is how Vagrant configuration file is a Ruby script with weird semantics around conditionals, which makes it its own special language to learn. They would have been a lot better off with the configuration approach taken by Xen (where the configuration file was a straightforward Python system).

Also, it's error messages are optimized too much for people developing Vagrant itself, and not enough for helping end users who are using Vagrant, which means one has to google often to figure out what the actual problem is.

Still, I don't think there's a better alternative for a development environment that Just Works for hundreds of developers. Docker isn't really designed for the development environment use case in my view, since it's optimized for throwing away state and getting a clean one when you make changes, and that's sometimes really not what you want. And having to SSH into a remote development environment has significant latency and editor setup costs that in my view make it a backup plan, not the main way to do things.

We use CasperJS because we adopted it back in 2013 for JavaScript frontend testing. It was a really nice system back then compared to what else was out there; you had PhantomJS as a programmable browser that actually rendered CSS and everything, it was really fast (speed is a big downside of e.g. Selenium), and it was possible to make non-flaky frontend integration tests with it.

I wouldn't recommend it today, because PhantomJS is a basically dead project, and as a result, so is CasperJS. I expect we'll migrate to something else. We haven't in large part because 95% of our new tests are written with a simple Node.js-based unit testing framework we use that run 35K lines of unit tests covering most of our JS codebase in 3.6 seconds. And for the things where we want an integration test, CasperJS does work, and I think there's a good chance that waiting another year or two will result in our being able to switch to a much better option than what we'd get if we migrated now.

We use Webpack because it's the standard toolchain for managing frontend dependencies in 2019, and it's hard to make a nice frontend development user experience without it.

I don't like it -- their configuration system is a mess, requiring a ton of reading or expertise to do things that essentially every project wants to do by default. It has a lot of great features, which is why we use it. But as an example, it's development server hot reloading is really cool, but doesn't handle changes in the webpack configuration file itself (so adding a new file requires a restart).

My hope is that the sheer fact that everyone is using it will eventually lead to these problems being fixed or it being replaced by a similar system with a better design.

We use Codecov because it's a lot better than Coveralls. Both of them provide the useful feature of having nice web-accessible reports of which files have what level of test coverage (though every coverage tool produces reasonably nice HTML in a directory on the local filesystem), and can report on PRs cases where significant new code was added without test coverage.

That said, I'm pretty unhappy with both of them for our use case. The fundamental problem with both of them is that they don't handle the ~1% probability situations with missing data due to networking flakiness well. The reason I think our use case is relevant is that we submit coverage data from multiple jobs (one that runs our frontend test suite and another that runs our backend test suite), and the coverage provider is responsible for combining that data together.

I think the problem is if a test suite runs successfully but due to some operational/networking error between Travis/CircleCI and Codecov the coverage data for part of the codebase doesn't get submitted, Codecov will report a huge coverage drop in a way that is very confusing for our contributors (because they experience it as "why did the coverage drop 12%, all I did was added a test").

We migrated from Coveralls to Codecov because empirically this sort of breakage happened 10x less on Codecov, but it still happens way more often than I'd like.

I wish they put more effort in their retry mechanism and/or providing clearer debugging information (E.g. a big "Missing data" banner) so that one didn't need to be specifically told to ignore Codecov/Coveralls when it reports a giant coverage drop.

We use ESLint because it's the best JavaScript linting toolchain available. I've been very happy with its level of configurability, though I think their choice of JSON for the configuration language is a particularly poor one.

The performance is also quite good in the latest versions; it's considerably faster the Python linters we use to lint our backend and most of the other ~10 linters we run in parallel as part of our normal linting experience (see linked blog post for details on that).

We use Plotly (just their open source stuff) for Zulip's user-facing and admin-facing statistics graphs because it's a reasonably well-designed JavaScript graphing library.

If you've tried using D3.js, it's a pretty poor developer experience, and that translates to spending a bunch of time getting the graphs one wants even for things that are conceptually pretty basic. Plotly isn't amazing (it's decent), but it's way better than than D3 unless you have very specialized needs.

We use Underscore because it's a reasonable library for providing all the reasonable helper functions missing from JavaScript ES5 (or that perform poorly if you use the default ES5 version).

Since we're migrating the codebase to TypeScript , we'll likely end up removing most usage of it and ultimately no longer needing it, but we've been very happy with the library.

We use Handlebars.js because it's a solid, performant JavaScript templating system with reasonable tooling. We adopted it more than 5 years ago now, and haven't felt significant pain points with it. Further, it's been easy to add i18n support and other useful extensions over time.

We might migrate to another templating engine if we migrate to a JavaScript framework with a built-in templating engine (basically because you have to), but I think it'd take something like that to make us want to switch to something else.