Avatar of Julien DeFrance

Julien DeFrance

Principal Software Engineer at Tophatter
Avatar of juliendefrance
Principal Software Engineer at Tophatter·

Back in 2014, I was given an opportunity to re-architect SmartZip Analytics platform, and flagship product: SmartTargeting. This is a SaaS software helping real estate professionals keeping up with their prospects and leads in a given neighborhood/territory, finding out (thanks to predictive analytics) who's the most likely to list/sell their home, and running cross-channel marketing automation against them: direct mail, online ads, email... The company also does provide Data APIs to Enterprise customers.

I had inherited years and years of technical debt and I knew things had to change radically. The first enabler to this was to make use of the cloud and go with AWS, so we would stop re-inventing the wheel, and build around managed/scalable services.

For the SaaS product, we kept on working with Rails as this was what my team had the most knowledge in. We've however broken up the monolith and decoupled the front-end application from the backend thanks to the use of Rails API so we'd get independently scalable micro-services from now on.

Our various applications could now be deployed using AWS Elastic Beanstalk so we wouldn't waste any more efforts writing time-consuming Capistrano deployment scripts for instance. Combined with Docker so our application would run within its own container, independently from the underlying host configuration.

Storage-wise, we went with Amazon S3 and ditched any pre-existing local or network storage people used to deal with in our legacy systems. On the database side: Amazon RDS / MySQL initially. Ultimately migrated to Amazon RDS for Aurora / MySQL when it got released. Once again, here you need a managed service your cloud provider handles for you.

Future improvements / technology decisions included:

Caching: Amazon ElastiCache / Memcached CDN: Amazon CloudFront Systems Integration: Segment / Zapier Data-warehousing: Amazon Redshift BI: Amazon Quicksight / Superset Search: Elasticsearch / Amazon Elasticsearch Service / Algolia Monitoring: New Relic

As our usage grows, patterns changed, and/or our business needs evolved, my role as Engineering Manager then Director of Engineering was also to ensure my team kept on learning and innovating, while delivering on business value.

One of these innovations was to get ourselves into Serverless : Adopting AWS Lambda was a big step forward. At the time, only available for Node.js (Not Ruby ) but a great way to handle cost efficiency, unpredictable traffic, sudden bursts of traffic... Ultimately you want the whole chain of services involved in a call to be serverless, and that's when we've started leveraging Amazon DynamoDB on these projects so they'd be fully scalable.

READ MORE
16 upvotes·2.2M views
Avatar of juliendefrance
Principal Software Engineer at Tophatter·

As a Engineering Manager & Director at SmartZip, I had a mix of front-end, back-end, #mobile engineers reporting to me.

Sprints after sprints, I noticed some inefficiencies on the MobileDev side. People working multiple sprints in a row on their Xcode / Objective-C codebase while some others were working on Android Studio. After which, QA & Product ensured both applications were in sync, on a UI/UX standpoint, creating addional work, which also happened to be extremely costly.

Our resources being so limited, my role was to stop this bleeding and keep my team productive and their time, valuable.

After some analysis, discussions, proof of concepts... etc. We decided to move to a single codebase using React Native so our velocity would increase.

After some initial investment, our initial assumptions were confirmed and we indeed started to ship features a lot faster than ever before. Also, our engineers found a way to perform this upgrade incrementally, so the initial platform-specific codebase wouldn't have to entirely be rewritten at once but only gradually and at will.

Feedback around React Native was very positive. And I doubt - for the kind of application we had - no one would want to go back to two or more code bases. Our application was still as Native as it gets. And no feature or device capability was compromised.

READ MORE
8 upvotes·8 comments·282.7K views
Peter Suwara
Peter Suwara
·
February 28th 2019 at 8:10AM

Generally speaking, on larger apps, React Native become unmanageable. I would be interested in hearing what kind of apps you are building. My enterprise apps collapsed when the the view structure become complex. Things like managing data and transports become convoluted and hard to follow.

We had much better results with Native and Rx. RxSwift has been a good way to bind our model to views. React Native became extremely difficult to maintain and was constantly changing and shifting, there was little stability, we dropped it and our performance improved as did morale, with less bugs and faster time to fix bugs.

·
Reply
Julien DeFrance
Julien DeFrance
·
February 28th 2019 at 3:12PM

We used React Native for a CRM-like type of application. Our users (real estate agents) paginating though contact records, editing their information, logging activities, taking pictures, visualizing them on a map, etc. Also, accessing their dashboards while on the go.

https://itunes.apple.com/us/app/smartzip-checkin-app/id988589730?mt=8

·
Reply
Julien DeFrance
Julien DeFrance
·
February 28th 2019 at 3:15PM

Some relatively large apps/customers are actually using #ReactNative :

https://facebook.github.io/react-native/

·
Reply
Ishan Fernando
Ishan Fernando
·
March 4th 2019 at 4:04AM

Isn't increase the app size? Or Does it same as a native app size?

·
Reply
Julien DeFrance
Julien DeFrance
·
March 4th 2019 at 4:57AM

I can't recall that part, however here's an article I found on the topic: https://medium.com/@aswinmohanme/how-i-reduced-the-size-of-my-react-native-app-by-86-27be72bba640

·
Reply
Avatar of juliendefrance
Principal Software Engineer at Tophatter·
Recommends
AlgoliaAlgolia

In the early days, people would set up their own Elastic Search clusters and have troubles maintaining it, keeping it secure, also this required a lot of manual work to get the data in, keep it current, query it... etc. A couple of years ago AWS came up with AWS Elastic Search Service, which reduced some of this overhead. My previous teams and I went through all of these different stages, and ultimately, discovering Algolia a couple of years ago, solved so many of our issues, kept the cost low, reduced dramatically Implementation therefore GTM timelines, and freed up so much of our engineering bandwidth. They have SDKs for most common languages and platforms, and you can achieve a complete solution in just a matter of hours if not minute. Value your own/your team's engineering time. Factor that in when comparing costs. There should also be entry-level tiers you can get a proof of concept rolled out with.

READ MORE
How Algolia Works | Getting Started | Guide | Algolia Documentation (algolia.com)
7 upvotes·373 views
Avatar of juliendefrance
Principal Software Engineer at Tophatter·
Shared insights
on
BootstrapBootstrapLessLessSassSass
at

Which #GridFramework to use? My team and I closed on Bootstrap !

On a related note and as far as stylesheets go, we had to chose between #CSS, #SCSS, #Sass , Less Finally opted for Sass

As syntactically awesome as the name announces it.

READ MORE
6 upvotes·67.3K views
Avatar of juliendefrance
Principal Software Engineer at Tophatter·
Recommends
RubyRuby

Ruby on Rails is far from being dead. In fact, this is a very popular choice in early-stage startups, given how fast and easily it allows them to launch their product and iterate on it.

Even at more mature companies, you'll still find a ton of opportunities. Not for internal tools or legacy codebases, but for actual production workloads: web apps, APIs, etc...

Some may tell you that Ruby doesn't scale, but is it really Ruby that doesn't scale, or the code they wrote?

Languages have trends. Sometimes, recruiters will try to take you one way or another to meet their own agenda. Don't always listen to what you hear. Long live Ruby! Long live Rails!

READ MORE
We Asked the Industry: "Is Ruby on Rails Dead?" | Netguru Blog on Ruby (netguru.com)
6 upvotes·1 comment·22K views
Matt Williams
Matt Williams
·
October 15th 2020 at 9:37PM

Thank you so much for the advice!

·
Reply
Avatar of juliendefrance
Principal Software Engineer at Tophatter·
Shared insights
on
RubyRubyRailsRailsGemfuryGemfuryGitGit
at

Working with Ruby on Rails also means working with #RubyGems Most of the time, the community has some gems you can use and list down your #Gemfile. But sometimes, you also need to come up with your own proprietary ones to encapsulate and re-use some of your business logic.

It is critical that such repositories and their source code remain private, secure. Even though your code shouldn't contain any credentials, this still applies to your gems' distribution channels. Unless for parts you've willingly open sourced, you don't want your intellectual property stolen.

Rubygems.org therefore, not being an option for this use case, I faced two alternatives: accepting the overhead of maintaining my own gem server, or finding a service that does it for me.

Obviously, the latter was the way to go:

I chose Gemfury for its convenience, pricing model, and reliability.

Gemfury also allowed me/my team to publish gems via different methods: file upload, SSH, HTTPS, or as simple as a Git push.

READ MORE
Gemfury (gemfury.com)
6 upvotes·10.3K views
Avatar of juliendefrance
Principal Software Engineer at Tophatter·
Shared insights
on
Swagger UISwagger UIRubyRuby
at

Use case: Keeping all API endpoints documented.

Swagger UI is slick. Not only details the specifications of all input/output parameters are there, but the interface also is interactive and allows sample requests to be sent to the actual endpoints.

With the help of Ruby gems such as https://github.com/richhollis/swagger-docs, the JSON files can automatically be generated for you for every controller you want to appear on the documentations page.

READ MORE
5 upvotes·42.6K views
Avatar of juliendefrance
Principal Software Engineer at Tophatter·

Hi Vibhanshu, When it comes to serving a static website, single page application, S3 or a combination of S3/CloudFront, is a great fit. There is no server that you need to manage, and S3 is as resilient and scalable at it gets. Moreover, it's certainly the most cost-effective solution you'll be able to come up with. Is your application fully static or does it come with compute needs? - Regardless, I would highly discourage you from running anything directly on a custom EC2 instance of your own, as this will from the get-go but also over time, come with high maintenance costs. - Elastic Beanstalk, (not to be mistaken with EBS, which acronym stands for Elastic Block Storage), on the contrary, is a rather powerful way to manage your applications and environments. Either via the CLI or through the console, you can easily configure your environment variables, load balancers, certificates, events/thresholds causing your instances counts to scale up/down, steaming of logs... - Elastic Beanstalk supports by default a couple of language, but these don't always allow you to run the version/dependency you need. For best decoupling from the underlying instances, you might want to look at leveraging Docker (Elastic Beanstalk has Docker-compatible AMIs), so you are in control of the stack you application runs on.

READ MORE
5 upvotes·3 comments·3.9K views
Kaibo Hao
Kaibo Hao
·
January 29th 2020 at 12:05AM

@Julien 's reply clearly explains the options to your requirement. While I would like to recommend Elastic Container Service(ECS) on AWS which provides more flexibility than EBS and less management on EC2 nodes. Please take advantage of Fargate which is a mini managed virtualization service in ECS two options (The other is EC2 instance) for the container host. The Fargate provides the container service with efficient cost.

·
Reply
Vibhanshu Biswas
Vibhanshu Biswas
·
February 10th 2020 at 7:36AM

I have mixed up EKS and fargate now. need to revisit the two technologies later on. or you might provide a better understanding. right now I have started the elastic beanstalk with docker.

·
Reply
Vibhanshu Biswas
Vibhanshu Biswas
·
February 10th 2020 at 7:34AM

I have started with Elastic beanstalk with Docker and it has made my task a bit easier. I will share more update as and when I proceed. Currently I need to decide what to do with my existing mongo db. we were managing this server on our own. I'm thinking of moving to a managed instance, again depends on the cost. the services I have in my mind is Atlas or the documentDB. I'm still assessing these two.

·
Reply
Avatar of juliendefrance
Principal Software Engineer at Tophatter·
Recommends
RailsRails

Rails is an easy framework to pick up, and you'll get to love all of the magic it does for you. Some of that can be a little confusing at first but once you've got acquainted, this is part of the productivity Rails offers as opposed to other languages or frameworks that sometimes tend to require developers to waste a ton of valuable time setting up their own boilerplate when starting to work on a new project. More pragmatically, Rails is still extremely popular at both startups and at large companies, you can use it to power web applications, or backend APIs, and this will be extremely valuable on your resume. There also is a very large/rich set of libraries (called gems) that will allow you to focus on your actual project/product, rather than rebuilding what already exists. I'd recommend you go with the latest versions of Ruby (3.0) and Rails (6.1.1) so you are from the get-go learning them in their most current form.

READ MORE
Rails 6.1: Horizontal Sharding, Multi-DB Improvements, Strict Loading, Destroy Associations in Background, Error Objects, and more! | Riding Rails (weblog.rubyonrails.org)
4 upvotes·1 comment·173 views
Julien DeFrance
Julien DeFrance
·
January 24th 2021 at 4:41AM

The IDE I use for Rails is called RubyMine, by JetBrains. You can try it free for 30 days. https://www.jetbrains.com/ruby/download

·
Reply
Avatar of juliendefrance
Principal Software Engineer at Tophatter·

Nexmo vs Twilio ?

Back in the early days at SmartZip Analytics, that evaluation had - for whatever reason - been made by Product Management. Some developers might have been consulted, but we hadn't made the final call and some key engineering aspects of it were omitted.

When revamping the platform, I made sure to flip the decision process how it should be. Business provided an input but Engineering lead the way and has the final say on all implementation matters. My engineers and I decided on re-evaluating the criteria and vendor selection. Not only did we need SMS support, but were we not thinking about #VoiceAndSms support as the use cases evolved.

Also, on an engineering standpoint, SDK mattered. Nexmo didn't have any. Twilio did. No-one would ever want to re-build from scratch integration layers vendors should naturally come up with and provide their customers with.

Twilio won on all fronts. Including costs and implementation timelines. No-one even noticed the vendor switch.

Many years later, Twilio demonstrated its position as a leader by holding conferences in the Bay Area, announcing features like Twilio Functions. Even acquired Authy which we also used for 2FA. Twilio's growth has been amazing. Its recent acquisition of SendGrid continues to show it.

READ MORE
3 upvotes·453.1K views