Operations Engineering @ ThredUP

By Syed Usamah

At ThredUp, our engineers are broadly divided between three teams: Mobile, Web Engineering, and Operations Engineering. In this post, I am going to talk mainly about what we do here as operation engineers.

ThredUp is a very operations-centric business, and the Ops Engineering team plays the vital role of providing the systems, tools, and the analysis required to make the company’s operations more streamlined, automated, and scalable.

That was a mouthful, so let me explain a little more.

When I joined the company in mid 2012, our warehouse was run entirely on homegrown iOS apps. It was quite the sight; everyone was walking around doing stuff with iPhones, be it receiving bags, or taking photos and measurements of garments to put on our online store. For our size then, the approach was working incredibly well. But soon, we realized that to achieve scale and higher efficiency, we needed something more robust. Thus, we intensified our focus towards building new systems to improve our operational efficiency.

While doing so, we have not restricted ourselves to any particular platform or language. We knew from the start that to solve such a wide array of problems, we needed a diversified technology stack. The result: our bag receiving is still done using a custom iOS app, our garments are photographed using a .Net station, their images processed using C++ and Python libraries, and the branding and categorization is done using a slick Backbone.js UI with touchscreens (mind you, this is just the inbound side of operations.) That said, our backend is primarily in Ruby on Rails.

Another distinctive feature of Ops Engineering is that  we do not have project managers or business analysts in our team; we ourselves do the managing and analyzing. For me, this makes for a really valuable experience, and I believe it both enhances and complements our teams programming abilities.  On one hand, working with the stakeholders directly to understand business requirements results in better and more suitable features. On the other hand, understanding the technical ins and outs of our systems helps us in explaining the functionality, tradeoffs, and limitations better to the stakeholders.

So what do we do on a daily basis you may ask? I have no straight answer for this. We could be hiding in a room, chugging coffee and coding, or we could be training the ops team on a new feature we just deployed, or we could be analyzing critical metrics from the last quarter and figuring out how we can make further enhancements. We are also not an all-work-and-no-play kind of a team. Some of us go work out together, others go to Meetups and conferences together, and others go tour breweries and discover random bars in SF together.

By the way, we are hiring for a number of engineering positions. If interested, check out our job postings.


Integrating your rails app with Dropbox

By Chris Homer - @chrishomer

At thredUP, we recently started using dropbox to do more than just store and share files among employees.  We've now integrated it with a couple of our applications so that certain data sets get archived there.  We used to just send them by email if they were small enough or save them to a server and then download them manually.  Now, with dropbox, they get dropped in a shared folder and the people that need access, have it.  

Implementing this integration had a couple hiccups along the way and I want to share how we did it in this post.

First, we used the dropbox-sdk gem (https://www.dropbox.com/developers/core/sdks/ruby) to handle most of the heavy lifting but you still need to implement a workflow to gain authorization with dropbox.  

We wanted to be able to authorize a conneection and persist the session to the database so we could restore it and work with dropbox without having to reauthorize again.  To accomplish this, we wrote a model for storing a users authorization session information:

As you can see, it's a fairly compact class. Here's how to use it:

With this wrapper class it's really that simple.  You could always implement a simple scaffold for this class and customize it to handle the auth workflow through the web browser - we don't add accounts that often, so didn't take this extra step.

A few more client use tips:

Hope this helps you integrate with Dropbox in your own applications!

A less permanent rails console

In a few of the previous places I've worked, it was common practice to use transactions whenever we needed to run queries on production environments.  What this does is give you the chance to verify that what you did was "correct".  In mysql, entering transaction mode is pretty straight forward:

Even when working in a developer environment, it's sometimes useful to have the ability to undo your work, so that you don't taint your data (which often takes a long time to reload if it's derived from a subset of production data).  

Here at thredUP, I tend to use rails console for prototyping a lot of the code I write (I know, I know, that's what tests are for) to validate that it's working as expected.  Recently, I ran into a situation where I needed to perform a destructive operation over a subset of data.  It becomes difficult to iteratively tweak or improve such code when the subset you're operating on is no longer there!  So I discovered a sweet flag when starting rails console that allows you to essentially make your entire session a transaction.

So now you can do any anything to your data and undo it by simply exiting rails console.  It appears this is done behind the scenes (watch the logs, to see how) by enclosing any queries called inside a begin/rollback block in mysql.  Interestingly, when you perform a save operation, rails saves the state prior to the write operation and then reverts back to it after the write.

For example, the following commands run on the console (in sandbox mode):

Produces the following log entries on the development log:

Notice the 2nd and 4th lines in the log, where it's saving its spot before the db write and restoring it after, respectively.  Finally, when you exit out of rails console, rails issues a ROLLBACK on the entire session.  You're back to where you started, which can sometimes be a godsend (especially in production :)).

From the Dev Couch: All Ruby Baby

by Dan DeMeyere - @dandemeyere

Rubyflow is a great Ruby blog aggregator. I subscribe to their RSS feed and every day there are multiple posts that either pique my interest or are relevant to gems or technology that I work with. I don't have time to read all of them everyday so I bookmark all the good ones and set aside a couple of hours every week to parse through them all. Featured in this post are the articles that I learned something from or determined are worth saving for a later date as it could become handy in the right situation.

Realtime Data Overlays Using Google Maps

The Ghostly Map
A couple of months ago the thredUP dev team wandered over to Cloudflare's office in SOMA to celebrate their recent achievements. While we were enjoying a couple of beers, we stumbled across a 10-foot projector screen displaying a realtime geo-mashup of their service in action. My colleagues and I all looked at each other and knew instantly we had to do something similar. While our version is still on our 'fun to-do list', this post on Stac's blog is a good walkthrough about how to make a realtime maps mashup with Rails 3. They cover using Rails' cache store API, use of Backbone.js, Underscore templates, GeoKit, and the Google Maps API. Read more →

Automatic Page Loading

Endless Page Scrolling with Rails 3 and jQuery
This has to be the coolest post I came across. Do you ever wonder how Facebook only displays a certain number of comments or status updates and when you approach the bottom of the container, they automatically add more? Well, this is how you can do it. It's a relatively simple implementation using jQuery to attach a custom handler on an element that makes an AJAX call to your Rails controller to append more data to the element. The only required file is his custom jQuery library, which can be found on GitHub here.

Altering Rails' Naming Conventions

Pluralizations and Singularizations (Inflections) in Rails 3
When I first started learning Rails, I found the naming conventions difficult in routes, controllers, and models. What was plural, what was singular, and what stayed the same - i.e. if the model is history.rb, is the controller histories.rb? In general, Rails is pretty good about getting that kind of stuff right, but if you want to override how they pluralize or singularize anything, then this post will be great for you.

Optimizing MySQL

Tuning MySQL - Innodb, Buffer Pool Sizes, and Rails Stack Server Advice
I'm not going to even pretend I know 10% of what's going on in this post. Even though it's way over my head, I have to imagine every person who works on a MySQL-based app can read this post and have some valuable take-aways. Did you know you can log slow queries by changing performance values in your MySQL my.cnf file? You can even change how many seconds constitutes a long query for your app. On top of that, you can log queries that aren't using indexes. Now that's pretty slick. There is also good advice for those using a Rails/Apache/Passenger/MySQL/Memcache stack (which is what we use at thredUP). Read more →

Notable mentions