How Homejoy Built A Global On-Demand Home Cleaning Service
Yonas Beshawred
Homejoy is often referred to as "Uber for cleaning services." But these days, they're moving beyond just cleaning, into handyman services, moving and storage, furniture assembly, and appliance repair. Their vision is to become the platform for any and all home services. Homejoy is now available in more than 30 cities throughout the US, Canada, Germany, and the UK - covering over 100m households. We sat down with Sid Doshi, Engineering Manager at Homejoy, to learn more about how they built their global on-demand cleaning platform and how they're scaling it today. Listen to the interview in full:
The really first version was mostly just Adora and Aaron, just building something very simple which would let people say, "I want a cleaning at this point, this place for this many bedrooms and this many bathrooms," and whatnot. Then, we had ... The service professional roster was so small that essentially it was just, "Okay, let's find out who can go."
It was all very manual...we had this web interface that was essentially mobile-sized, iframed into a full sized webpage, so it was kind of our mobile site and our main website at the same time.
We were on App Engine mainly for historical reasons and yes, I think it allowed us to scale very, very quickly. App Engine used to work with the Google Data Store. So we were on NoSQL. Our data was very, very relational though...We started running into more issues along those lines, especially around payments and around scheduling where we might double schedule someone or especially around peak times and a lot of bookings are happening...
So we took the App Engine SDK.. and we kind of changed where the data went from the datastore to a PostgreSQL instance on AWS. So right now the way the application works is, it thinks it's an App Engine application. We use the same libraries. In most parts still, or at least for the old code. Then it sends this data to PostgreSQL instead or fetches it from PostgreSQL.
Ultimately we're providing this layer that enables a customer and a professional to work together and we want to try to automate that as much as possible to make it as efficient as possible.
The more time we take on it and the more manual that process is, we're spending unnecessary resources and time, that we don't really need to...Inside the company we do have a separate team that's focused on non-cleaning services that we do right now...I think that there's some very exciting stuff that we're building from a data analysis perspective. We're building a lot of stuff around bringing in data from different sources and exposing the data to the right people in the right way, just giving us sort of immediate insight into things on an ongoing basis.
Homejoy
Homejoy MVP
Python
+
webapp2
+
Google App Engine
+
Google Datastore
Background & MVP
Yonas: All right. Thank you Sid. Thanks for doing this.
Sid: Absolutely. Thanks for having me.
Yonas: Do you want to just start with a little bit of background on yourself?
Sid: Sure. So I'm a mechanical engineering grad. I was doing grad school in Georgia, in Atlanta at Georgia Tech. While there, I was basically looking at urban transportation as a complex system. One of the big things that was coming up back then was bike sharing. This was 2009-ish ...
Around 2010, me and a couple of people from my lab at Georgia Tech, we decided ... Well bike sharing, the way it existed was a little bit complex to implement and a bit too expensive. So we thought we could do it better. We started this company called viaCycle. What viaCycle did was, we made bike sharing systems, not to sell to individuals, but more to provide technical systems to bike share operators.
We were doing hardware and software connected devices around bicycles. We had this mechanical unit that we designed that went on bicycles and was kind of this automated lock system that we could control from our servers. I did that for about three years I think. We started in late 2010 and went up to around mid-2013.
That was what I was doing, before Homejoy. It was definitely a fun experience. We had some paying customers. We had some bike systems running, specifically on, mostly on university campuses and so on. We did YC with that in Summer 2012, also a fun experience. Ultimately, it didn't turn out to be the market we hoped it would. Like a lot of start-ups, we decided that, okay, this isn't really where we want to be. We shut it down in 2013 and I actually, sort of literally put everything I had in my car and drove up here from Atlanta.
Y: Wow, okay.
Sid: Then I found Homejoy. It’s definitely been a great experience since I joined. I've been at Homejoy for about a year and a half now. I am an engineering manager here. I mostly work these days on workflow optimizations, building our tools that we use internally to manage Homejoy. I worked on pretty much all parts of the stack back end, on devops. Bunch of different places at different times.
Y: Maybe we can get a little bit into the background for Homejoy itself. A lot of people have called you Uber for home cleaning, Amazon for home services because now you're doing things beyond just home cleaning. Can you talk a little bit about Homejoy and the vision here?
Sid: Homejoy, what we want to be is "the get help button for the home." Essentially, you need something done in your house. We want to make something where you can just go to Homejoy, say, "I want this," and we can just arrange everything online. It's really, really hard to get help for a lot of things these days. You go to Craigslist or essentially, it's a lot of unorganized stuff and what we want to build is this platform and this brand that you can trust. You can come here and say, "Okay, I'm going to get some Homejoy. I need something done at my house."
Definitely I think "Uber for cleaning" is a good way to sort of ... It was definitely a good way to describe Homejoy at the beginning. It’s just this sort of this way to immediately understand what Homejoy did, but I think the vision is definitely a lot larger than that.
Y: What are some of the other services you guys offer now?
Sid: Specifically, in San Francisco, we do handyman services, carpet cleaning. We tried laundry for a while. We decided we're going to come back to it in a bit. We've been playing around with a few services, but right now we offer handyman and carpet cleaning.
We're in about 30 markets across the U.S. and London, Berlin, Munich, Hamburg.
Y: You have a lot of cleaning professionals and people on the supply side, right?
Sid: I'd say we're close to around 1,500 cleaning professionals maybe on the platform.
Y: Awesome. I guess we can start off just by talking about maybe the first version.
Sid: Sure.
Y: What was the "just get something working" version?
Sid: I would say ... I'll go back to the first version that I know of. Then obviously ... Then I came in. We were sort of on version two, but I know of the first version, which is, I would say, more of a proof of concept. The really first version was mostly just Adora and Aaron, just building something very simple which would let people say, "I want a cleaning at this point, this place for this many bedrooms and this many bathrooms," and whatnot. Then, we had ... The service professional roster was so small that essentially it was just, "Okay, let's find out who can go." It was all very manual. I would call that more of a proof of concept than an MVP.
Y: Was it a mobile app or was it just like a web interface?
Sid: No, it was a web interface, though the interesting thing about that was we had this web interface that was essentially mobile-sized, iframed into a full sized webpage, so it was kind of our mobile site and our main website at the same time.
Y: Got you, got you. Okay. That makes sense.
Sid: It is kind of an interesting spin on mobile-first development, but whatever works. That was kind of the really early version. We built a lot of the backend out after that, but we used that for a while there. Essentially you'd go to the website and it would be this mobile version or this mobile webpage that was inside of a webpage.
Y: Right. What were you using initially?
Sid: From the very start to up until maybe, I'd say eight, nine months ago, we were on Google App Engine. Google App Engine is kind of this, for anyone who's not familiar, is this platform as a service offering from Google that's similar to Heroku if you're familiar with that. So we were on Python on Google App Engine.
That meant we used their SDK for the ORM part of things. Then on the front-end, we were mostly on just basic JavaScript and HTML.
Y: Got you. Okay, all right. That was the MVP.
Sid: Well, yeah. I would almost call it sort of a proof of concept and/or an MVP, but that’s kind of what we started with. Then we went from there to having a more regular website.
Y: Right, right. Early on, what were some of the biggest challenges?
Sid: Some of the biggest challenges back then would have just been scaling constantly. We were growing very very fast and it was always ... You start with an MVP, right? You're always building for almost exactly what you need or just more than what you need which means as you add more people in the system, things start breaking pretty quickly. Especially in the early days, we were always racing against this, okay, we're growing fast. This used to work at a lower scale. This is not working anymore so let's keep fixing it. At the same time you're not necessarily over-engineering things so we can keep moving faster.
Specific example of that would be our scheduling algorithms. Very early on it used to be literally just this ... You would go through the entire list of professionals that you had and figure out who was available at a particular time because the list wasn't that big. Simple to set up, code and just kind of get out there. Of course, as you go beyond a certain number and then that becomes a not very viable way to do things and so we moved to more advanced scheduling algorithms and then those broke and then we went to better stuff there.
That, I think, has always been sort of our major challenge. Especially in the early days, that was a big challenge with limited engineers available. We had about six engineers. We had Mark who's the VP of engineering now and then one front-end person and then basically five full-stack developers.
Backend & Infrastructure
Homejoy
Python
+
Flask
+
PostgreSQL
+
Amazon EC2
Y: Let's talk about the current architecture. Now you have web and mobile.
Sid: We do have web and mobile. We also have some apps so definitely very different from those days. We're not on App Engine anymore. We migrated about 8-10 months ago.
We can talk about that. It was a pretty interesting shift for us, but just kind of cover the stack right now, we're on AWS, still a Python application ... we're sort of in the middle of transitioning, or ever since we moved off App Engine, we've been in this sort of two-app system where we have the old app that we migrated as is with this sort of translation layer in between, and then we have the new app that we're porting things over to as we move on.
The old app we still use more or less the same code that we used on App Engine. That means for the object relational part of it, we use the App Engine SDK. We use this application, this Python framework called webapp2 which is this really lightweight Flask-type application that people use on App Engine.
On the back end then, on the database side we have PostgreSQL and this all works because we have this translation layer in between, a shim, which essentially takes all the App Engine stuff and then converts it for PostgreSQL instead of going to the App Engine data store.
That's our current application. On the front end side, still fairly similar. We've built up some more stuff in terms of an asset pipeline and managing CSS, we use Stylus as a preprocessor and all that, but we aren't using AngularJS or Backbone or any of those sort of front-end frameworks. We are still mostly on plain jQuery and JavaScript and HTML/CSS.
Y: Okay. You said two apps right now?
Sid: Right. That's kind of our major app that we brought over from App Engine and then the new one we're developing essentially getting rid of the translation layer. That's going to be, or that is, fairly bare-bones right now but the way we're designing it, it's going to use Flask and then our own in-house object relational mapper so not so much ... Basically very simple. Kind of gives us direct access to PostgreSQL and lets us do complicated queries but then also abstracts away some of the complexity.
It's a much simpler way of doing stuff, but not so heavy on the framework side, heavier on just letting you do what you need and getting out of the way.
Y: Right, right. Okay. Do you want to talk about the different pieces in terms of how they fit into the experience? You have ... The main app is serving up both the web app and the mobile side.
Sid: We have a web app and we have the mobile web experience. This whole infrastructure that we just talked about, that serves up both of those and then it also serves up the APIs for the native mobile app so we have one on the iOS App Store right now. We don't have one for the Android Play Store, but that's in development. We have, on the professional side, on the cleaning professional side, we have an Android app that's very fully featured and that's essentially now the main way that they communicate with us so that also kind of ties into the APIs.
Y: That's interesting. Why have did guys choose to have a native Android app for the cleaning professionals and not on the consumer side?
Sid: We just found what our user base was and the main way people accessed our mobile websites and we sort of tackled those first and then basically left the other platform as a second one and we are sort of starting to build both of those as well, so Android on the client side and iOS on the cleaning professional side. Those are both in the pipeline, but we wanted to get what would cover the most number of users first, right.
For professionals, the big difference was we, we actually did this program where we rolled out phones from Homejoy to all of the professionals so we could focus on one platform, and so that was Android phones and that made sense for us to do the Android app first just as, that's what everyone was going to use.
There's a lot more choice there as well and also I guess as a platform there was more that we could do with Android than with iOS.
Y: Okay. Does that cover most of the experience? People can book through the website, right?
Sid: Yes. You can book through the website, you can book through ... There's a full-featured mobile web experience and then there's the apps.
Y: Okay. Where do you guys find most of your users coming in? iOS?
Sid: No, that's ... We actually, most of our users still use web. Mobile is coming up very quickly though, so that is a major focus for us going forward.
Y: Yeah, because it's for your home so you don't necessarily need to be opening up your phone ... Okay, that makes sense. Do you want to talk a little bit about your switch from Google App Engine to AWS?
Sid: Yeah. That was one of the more interesting things I think we've done here. We were on App Engine mainly for historical reasons and yes, I think it allowed us to scale very, very quickly. So no knock against App Engine, we loved it when we were using it. App Engine used to work with the Google Data Store, or the Google Cloud Data Store which essentially is an implementation of BigTable which is their NoSQL database. I think now they have an option of using a SQL database backend as well but back then it used to be only NoSQL. So we were on NoSQL. Our data was very, very relational though, so we had these ... with NoSQL what you ... or with BigTable at least what you didn't get was consistency. You got eventual consistency.
A lot of things, when you need it... you would put something in the data store and then it would be a little bit before ... it would be a few seconds before you knew that you could definitely get back the data you just put in. You might get an older record before that. So that causes problems with certain things. For a lot of things, it's fine.
We started running into more issues along those lines, especially around payments and around scheduling where we might double schedule someone or especially around peak times and a lot of bookings are happening. Just the type of company we are, our data was very, very relational so it kind of didn't necessarily make sense to use NoSQL, or the additional flexibility we got there wasn't necessarily very useful to us.
Y: But they do have a SQL store, right?
Sid: They do now.
Y: Oh right, at the time they didn’t.
Sid: We could have moved from App Engine NoSQL to SQL but that would have been almost the same amount of work.
That was kind of holding us back. I think it slowed down a lot of our systems as well. You'd go to the website and we would be querying the data store for availability of cleaners, right? And it would load for a couple of seconds and then show you what times are available. A couple of seconds is a lot, right? That was after you had to do hacks to optimize it and so on. We kind of saw the writing on the walls and yes, we need to move to a SQL relational database. It just made sense from a lot of different standpoints.
So then the question was, how do we do it? Do we start moving parts of the system or to a completely new system or do we just sort of find a way to move the system we have? It turned out that the second one was a good alternative and the way we did that was, as I was describing earlier, we kind of wrote this layer that goes in between. We took the App Engine SDK which is, the SDK itself is open source, so we had this Python library and there was code ... We kind of changed where the data went from the datastore to a PostgreSQL instance. So right now the way the application works is, it thinks it's an App Engine application. We use the same libraries. In most parts still, or at least for the old code. Then it sends this data to PostgreSQL instead or fetches it from PostgreSQL.
It’s a lot quicker, plus we can do a lot. We're more flexible with what we can do. We can do joins across tables. So when we bypass the App Engine SDK, we can do things that we couldn't do before.
Y: Okay. When you were going from Google App Engine to AWS, you could have gone with another platform as a service, right? Why did you guys decide to just go straight to EC2?
Sid: I think we were at the scale where owning our stack made sense.
Y: Okay, so it wasn't just the data store issue.
Sid: Right. That was the driving factor. Obviously this was a lot of work and the project went on over months, not for the entire team, but a lot of people worked on it and mainly one person worked on it for a long period of time.
We have an awesome DevOps engineer who built most of the translation layer. It was a big project and kind of a risky move too. I remember when we moved it over to the new architecture after testing it and retesting it for a while. We were all here at night. There were ten of us here at night just waiting and making sure everything went well and ...
It ended up being really smooth, actually. We ended up just hanging around waiting and not doing a lot. It was a scary move and a big move. The site could go down. We could have lost data and so on.
There were different reasons and the data was the primary reason.
Y: It sounds like since you were going to go and do a lot of rewriting then it made sense to also just move ...
Sid: Right. If we were gong to make changes anyways then it made sense for us to just own our stack at that point because ultimately Platform as a Service offerings are great to help you scale really quickly but there are limitations just because the platform is owned by the PaaS company and they decide what's available on it and what's not. There's some flexibility but it's limited and once you're on your own stack, then you can add remote tools from your stack as you want.
Y: Got you. Okay. Do you want to talk a little bit about the different pieces moving around here? We've got the main app in Python, right?
Sid: Yeah. And Flask on the new application, webapp2 on the old application.
Then we have Objective-C for the iOS app and then of course Android and Java on the Android App.
Tools & Workflow
Homejoy
Python
+
Flask
+
PostgreSQL
+
Amazon EC2
Y: Okay. Do you want to talk a little bit about some of the utilities and things you might be using inside the app?
Sid: Sure, yeah. Yeah, we use a bunch of external services. For email we use SendGrid and they've been very great. We recently, or relatively recently, started using sendwithus which manages ... that's where we manage our email templates now. The email still gets sent via SendGrid but sendwithus is more of an email template management solution.
We trigger emails from our application and then we tell sendwithus, "Send this template to this person," and then they send it via SendGrid. It's essentially, for some of our emails, especially the fancier HTML ones, it's a step in between us and SendGrid.
The utility it provides is it gives us this easy way to manage multiple templates and fancy templates that engineering necessarily does not have to own so we can have design or marketing handle the email template part of it, right.
For other services, we use Twilio fairly heavily for all our text messaging. When the jobs start. Basically reminders when the jobs end to get feedback and so on. We’re also using them. We have this system that allows us to connect our cleaning professionals with their customers when the job is upcoming and allow them to communicate directly without exposing the phone number, so that's goes via Twilio as well. They can call as well and that's routed through Twilio.
We’ve got Mixpanel for analytics. We also use BigQuery, are starting to use BigQuery a lot for event analytics, so just kind of streaming event data into BigQuery, besides other data that we pull in there.
That's sort of our data pipeline. We use Tableau to visualize a lot of that data from BigQuery. That's our data and analytics.
Y: All right. Maybe we can get a little bit into work flow, engineering work flow.
Sid: Sure. We use GitHub very heavily, obviously for source code management but also for code requests and code reviews as well.
Y: You use anything else for code reviews like automated code reviews?
Sid: No, mostly just GitHub. Well, we do linting automatically on all code requests so we have a Jenkins set up that we use for testing and just sort of code linting and so on. Every time we deploy we launch a process on Jenkins that runs all our unit tests and then also runs a linter on all the code.
Y: Okay, cool. Let's see, what else did we not cover here? Anything else you can think of that is just stuff you want to touch on inside the main app?
Sid: Yeah. I think we've covered most of it. Certain cool things that we're doing in the stack, obviously scheduling, I think, is a major part for us where figuring out what the best person ... Who's the best person to put on a certain job out of who's available, but then also ...
Y: Right, so, sorry to interrupt but in terms of matching people, are you taking into account metadata in terms of how fast people work and where they are and that type of thing?
Sid: Not so much how fast people work because that's sort of a subjective ... That's really hard to quantify, I think. We do take into account things like where they are, so how far they would be potentially. We also take into account things like do they have a car or not, so depending on the city and where you are, whether certain people can work with dogs or cats or have allergies so we'll take that into account. Yeah, so that type of thing.
And then there's also a lot of ... One of the big things that we take into account is whether someone was requested specifically for a job or not. For example, if you have had Homejoy cleanings, I'm not sure if you have.
Y: Not yet, not yet. I think we need it though, in our apartment.
Sid: Well, when you do get one, I think, you get the first one and if you like your cleaning professional and they did a really good job then you generally want them again, right? You would request the same person again and we do allow you to go to the website and say if you like this person, you can specifically book an appointment with that person. If that's true, then obviously that means that they are the one who are going to be assigned. That means you can't then play around with that schedule a lot. That's just going to be there.
What we do try to do is increase the utilization and try to match, or even move around some schedules, not in terms of moving the appointment but moving certain professionals around just to get the most number of jobs fit into a particular schedule so they're maximizing the time that they're working and not too many time in between jobs not doing anything.
Y: Right. That's interesting because I know some of the other services like Uber, you can't actually request a specific person. Sounds like you guys just made a decision, say, if you like someone, you should be able to ...
Sid: Right. For us, obviously that makes a huge difference because you're letting someone into your house, right? Once you build up trust or rapport with a particular person, they know how you like certain things, you don't want to keep giving the same instructions over and over again. That's where I think the big difference between us and Uber. Uber, it's five minute, or maybe a 15 minute transaction generally, right? Someone comes and picks you up, they drop you off, that's it. You generally never need to see that person again.
We are sort of ... When you book a cleaning, that's generally at least two and a half hours. That's the minimum you can book for, but sometimes much longer. It's maybe five, six hours sometimes. When it's a longer transaction like that, you obviously want someone that you feel comfortable with and you're letting them into your house so there's obviously this ... If you can't trust someone then it's a lot harder to let someone into your house, right?
Engineering Challenges & What's Next
Homejoy
Python
+
Flask
+
PostgreSQL
+
Amazon EC2
Y: Cool. Current challenges. Do you want to talk about just some of the engineering challenges you guys are facing and how you plan on addressing some of them?
Sid: Sure, yeah.
Y: Let's say top of mind for you, maybe not today, but in general.
Sid: There's a lot of stuff with workflow where we obviously, on the business side, we change how we operate as we learn more about what's needed, right? We change policies, we change how we do certain things in terms of what works better to onboard cleaners faster. What works better to communicate them with an ongoing basis. What works better to ... How do we deal with customers who get cancelled on at the last minute. These are all things that happen in a business. We try different things. We change that on a regular basis.
On the technical side, we need to sort of keep up with that and we want to build not just, okay, here are the tools, make all the changes you want to this appointment or to this customer profile. We try to be a little bit smarter than that and say, "Okay, these are the most common use cases. How do we make our agents ... Our agents are people who are doing these things inside the office, are managing these processes. How do we make their jobs faster and easier." Because ultimately we're providing this layer that enables a customer and a professional to work together and we want to try to automate that as much as possible to make it as efficient as possible. The more time we take on it and the more manual that process is, we're spending unnecessary resources and time, that we don't really need to.
That's kind of an ongoing thing.
And sharing data is another area. We do have tools to share data now. BigQuery allows us to do a bunch of that, but not necessarily with visualizations and that's where Tableau comes in.
You can definitely see stuff in Tableau as well so it's ... We're trying to make this data pipeline where you can, depending on your level of comfort with doing these things you can either go in at the query level and just build queries or you can go in at the visualization level and build visualizations, or people can just send them to you as well and you can just upload the data or whatever.
Non-engineers are a big part of Homejoy, right? Engineering is one part of Homejoy and obviously operations and how the other teams do things matter a lot to the success of the company. What we're trying to do is build the tools that allow non-engineers the level of access that they need without jumping through hoops or waiting for engineering resources, especially on the data side where just having access to, or having easy access to data allows them to make the right decisions without waiting around for someone in data science or someone in engineering.
Y: Yeah. Instacart actually built a really cool tool for that called Blazer. I’ll make sure to send it to you after this. But cool. So then there's the obvious challenges with how you guys are expanding beyond just cleaning, right? It's basically like you're building, not the entire system from scratch, but a lot of the things, you're going to have to redo, right?
Sid: Correct.
Y: Like the process for getting a handyman into a house. How are you guys thinking about that and do you have separate teams for specific types of jobs?
Sid: Inside the company we do have a separate team that's focused on non-cleaning services that we do right now. I don't think that's necessarily how things will always be, but this allows us to focus more on ... These are some ideas that are forming.
Y: Like your R&D.
Sid: Right. It's kind of this thing that can move faster and make changes as we see fit.
Y: Okay, got you. It terms of what's next, is it more just building out some of these new services or is it, again, just focusing more on automating as much as possible?
Sid: It's a lot on the services, a lot on the automation side, a lot on the mobile as well, so definitely ...
Y: Oh right, Android is coming.
Sid: Android is coming, iOS, also a lot of improvements there coming soon. Also on the professional side we have iOS needs to happen soon and then of course we're always adding more features and making it easier for us to communicate there on the Android side as well.
Y: Are you going to remain Python for the backend?
Sid: At least for the foreseeable future ...
Y: Okay. Do you have any questions Cyrus? Anything you want to ask?
Cyrus: What are you guys most excited about? What tools are you most excited about?
Sid: Sure. Yeah, so tools that we're building. I think that there's some very exciting stuff that we're building from a data analysis perspective. I was talking about the data pipeline earlier. We're building a lot of stuff around bringing in data from different sources and exposing the data to the right people in the right way, just giving us sort of immediate insight into things on an ongoing basis.
More than ... Since were talking about internal or external tools. That's definitely ... We're very excited about the data pipeline and of maturing it to a point where we can get ongoing insight into things. Right now what happens for a lot of the deeper insight stuff is someone makes a request to the data team, they run the analysis once and then we get the results and then we have that. With the new pipeline, this will allow us to do it once, keep it there and then look at it on an ongoing way, in an easy way.
In terms of external tools, there's a bunch of tools that we're using right now, but I think the focus there for me is good integration into a lot of external tools. There's a lot of cool stuff that we're doing on the customer support side where we're integrating with phone systems, with text like we talked about earlier. We're also looking at on the supply chain side, on the email side. There's just a bunch of different things we're doing there with it. We use Zendesk for example and we're looking at deeper integration there.
On the text side we actually use this app called Front for customer support as well which we're looking into for deeper integration as well which has worked fairly well for us so far.
Sid: Engineering. Well, across the company but for engineering. Also we're looking at mobile, Android, iOS, full stack, back end, front end, all of it.
Y: Cool. Awesome. No more questions? All right, well thanks a lot for taking the time.
