Feed powered byStream Blue Logo Copy 5

Decision at SendGrid about Buildkite, GitHub, InMemoryDatabases, Databases, DataStores, Languages

Avatar of sethgrid
Principal Software Developer at SendGrid

When we've gone through code reviews (every code and config change goes through a code review) and feel good about the level of automated testing (no one can sign off on their own code's functionality; a quality assurance engineer or other developer has to verify functionality), we merge our changes via a bot that interacts with GitHub (the bot maintains our versions and change logs). After Buildkite has a green build and our binary is shipped to our repo servers, we are good to roll out deploys to our data centers and to keep pushing the needle on the performance of our system.

#Languages #DataStores #Databases #InMemoryDatabases

17 upvotes1.5K views

Decision at SendGrid about Docker, BuildTestDeploy, VirtualMachinePlatformsContainers

Avatar of sethgrid
Principal Software Developer at SendGrid

For the unit-integration layer that tests transactional emails, we leverage Docker. Our incoming edge is when the upstream service is finished processing a message and hands it to us for delivery, and then our outgoing edge is actually communicating with someone's inbox. We don't actually want to set up a bunch of receiving MTAs and such, but we still need to test behavior at that layer. Our solution is still a work in progress, but it gets the lion's share of use cases covered so we can confidently refactor and push new features and know we did not break anything.

This Docker setup leverages DNSMasq for setting up MX and A records and ensures they point to running mock inbox sinks. These inboxes are configured from a base image with multiple options. We can specify that the sink's TLS certificate is expired or improperly set up, we can have them respond slowly or with given errors at different SMTP conversation parts. We can ensure that we are backing off and deferring email if the inbox provider says to do so. This detailed faking of the outside world allows us to automate all kinds of outside behavior and ensure that our services behave as expected.

We develop locally in Docker, as we just went into. Our docker-compose file spins up containers with fancy DNS settings and all our dependencies, allowing us to test the MTA against a variety of MX and TLS settings, alongside a variety of potential inbox responses and behaviors. Everyone uses their editor of choice and we often pair up on more complex tasks to prevent siloed system understanding.

#VirtualMachinePlatformsContainers #BuildTestDeploy

12 upvotes718 views

Decision at SendGrid about Go, Perl, Docker, ContinuousIntegration, CodeCollaborationVersionControl

Avatar of sethgrid
Principal Software Developer at SendGrid

In addition to our fancy Docker setup, we have captured and sanitized production logs for the behavior of our legacy Perl MTA, and we can test that the log output from the new Go version behaves the same way as the old version. These tests are set up to allow us to switch between the legacy and new version of the MTA and ensure that both systems behave in a legacy-compatible way. Not only can we ensure that we operate against a variety of issues we've seen over time from inboxes, but we know that the newest version of our MTA continues to cover all the same expected behaviors of the legacy version. #CodeCollaborationVersionControl #ContinuousIntegration

8 upvotes4.2K views