Apache Kafka at Spreedly: Part I – Building a Common Data Substrate
At Spreedly we’re currently undergoing a rather significant change in how we design our systems and the tooling we use to facilitate that change. One of the main enablers for this effort has been Apache Kafka, a distributed log particularly useful in building stream-based applications. For the past several months we’ve been weaving Kafka into our systems and wanted to take the opportunity to capture some of our thinking around this decision, how we decided to approach the project, and what we learned (and are still in the process of learning).
This is less an Apache Kafka tutorial – there are plenty of those out there already – and more a discussion of why Spreedly chose Kafka vs. other messaging systems like ActiveMQ or RabbitMQ. What specific use-cases did we have and why does Kafka make sense for us? Of course, a conversation like this will necessarily include a recap of various Kafka architectural details, but the intent is not to get weighed down in low-level details. From this, we hope you can apply what we’ve learned to make the most pragmatic choice for your organization.
Performing a Content Audit — A deep dive into Spreedly.com
I came on-board in April of 2016. Post on-boarding process and adjustment to the new role, preliminary design research began. Payments was a new industry and so was Spreedly. Managing the marketing website became a responsibility. It was time to learn a few things. After speaking with colleagues the path was clear: perform a content audit . And so I did.
If you’ve never heard of a content audit before, or have but don’t know what one is, no biggie; it’s pretty simple. A content audit is literally an audit of your content. What that means can vary but in most cases it’s how many pages of a site, what keywords it ranks for, usage of
h1/h2 tags, how many images, etc. Why you would perform a content audit depends upon your role and goals. Marketers may look at keyword performance, content creators may need to develop a content strategy, or designers may be improving the user’s experience.
Being as a content audit can be performed by different people for a variety of reasons, I decided I would look at things deemed beneficial to the role of designer at Spreedly. I needed to learn the nuts and bolts of the system I was working with and get an understanding of what I would/could do moving forward. Up-front there weren’t any initial assumptions of what I would find; I kept that open since this was to be an exploratory process.
Mocks and Explicit Contracts: In Practice w/ Elixir
Writing tests for your code is easy. Writing good tests is much harder. Now throw in requests to external APIs that can return (or not return at all!) a myriad of different responses and we’ve just added a whole new layer of possible cases to our tests. When it comes to the web, it’s easy to overlook the complexity when working with an external API. It’s become so second nature that writing a line of code to initiate an HTTP request can become as casual as any other line of code within your application. However that’s not always the case.
We recently released the first version of our self-service debugging tool. You can see it live at https://debug.spreedly.com. The goal we had in mind for the support application was to more clearly display customer transaction data for debugging failed transactions. We decided to build a separate web application to layer on top of the Spreedly API which could deal with the authentication mechanics as well as transaction querying to keep separate concerns between our core transactional API and querying and displaying data. I should also mention that the support application is our first public facing Elixir application in production!
How do I GenStage?
I’m new to Elixir. And Erlang. And OTP’s GenServer. And GenStage. While I’ve got beginner’s-eye, I’m going to share knowledge for a general audience. So this is a doorknob-simple look at GenStage. With clear examples and I Love Lucy references. Enjoy!
Purposeful Web Design — Designing the Spreedly Support App
As the Lead Designer at Spreedly I am fortunate (and excited) to work on many different types of projects. I was recently involved in designing our new Debug App. It’s a place where users can find answers to common support issues and do some basic debugging on their financial transactions. In this article I share insights into my process and some of the steps that were involved along the way.
I am a big fan of the design methodology behind Jeff Gothelf’s Lean UX. If you haven’t read it - check it out. Short read, easy to understand, and has a lot to do with my thinking behind the process that went into designing Spreedly’s self-service debugging app.
Stop Hazing Your Potential Hires
As an employer, it’s easy for an organization like Spreedly to complain about the difficulties of the hiring process. Huge investments of time with uncertain results, noisy pipelines of dubiously qualified candidates, lying recruiters, spamming spammers who can’t read, lack of good science around hiring processes and their effectiveness, etc. Isn’t an employer’s life tough?
But it’s all just whining, because you know who has it tough? Candidates applying for jobs have it tough. They have to put their whole work history in front of a total stranger, invest time and money with no guarantee of a favorable outcome, make a series of decisions that will probably affect the rest of their life, do right by existing commitments while trying to impress us, and at the end of the day get a binary “yes/no” judgement of their ability to do what they believe they’re good at doing, all with little to no recourse or appeal. It makes me tired just thinking about it, but that’s the reality of what it means to go looking for a new job.
You're Smart Enough for Elixir
A brief glimpse into Elixir from an object oriented perspective.
Concurrency is complicated so Elixir must be complicated.
Learning functional programming means giving up all my knowledge about object oriented programming.
Why Elixir? I haven’t needed it so far. It’s probably only for really complicated computer science-like stuff that I’m not doing.
If you’re looking at Elixir from the perspective of an object oriented scripting language like Ruby, Python, PHP, or even Perl then you’ve probably read, heard, or made statements like those above. Even if just to yourself.
I’m here to tell you that I’ve been there. I’ve professionally programmed in each of those languages over my career. Now I’m coming to Elixir from a recent background in Ruby. Let me assure you — you got this!
Programming Puzzles Are Not the Answer
A guide to creating fair and effective hiring work samples.
It’s a common topic of discussion amongst technical organizations that the traditional, in-person, conversation-based, interview process for developers is flawed. The conventional process does not measure a candidate’s ability to perform the job in question, does not result in more successful hires, and allows for implicit bias. As a result, many companies use an interview process based on programming puzzles: one-time exercises that test a candidate’s ability to solve a specific, well-known problem.
While this may be a step in the right direction, it’s still a hugely biased and ineffective approach to hiring. In this post I explain how we do hiring at Spreedly and, specifically, how we measure a candidate’s competence using job and company specific work samples.