Spark Connecting to Remote Spark Clusters I've been using Spark pretty much daily for about 3 years now. Its our main data analysis engine over at Coupa. Our code base has grown considerably over the years
Personal Software I Pay For As I've gotten older, I've found myself more willing to find and pay for software that provides value to me. Recently I decided to revisit some pieces of software I
Data Predicting Boss Outcomes in World of Warcraft World of Warcraft is a massively online multiplayer role playing game (MMORPG). One of the features of the game are these scripted boss encounters where anywhere from 10 to 40
Spark Spark best practices on EMR The default settings for Spark on EMR are… sub-optimal to say the least. This is meant to be a quick guide into some things I’ve found to be useful with provisioning a Spark cluster on EMR. Background Info Not all users have the
Detecting the game being played on a live Twitch stream This is part 3 in a series on detecting games on Twitch. You can read Part 1 and Part 2. Now that we can detect a game pretty reliably and fast, the next step is to see if we can apply this technique to
Machine Learning Replacing Google Prediction API In a previous post, I was using Google's prediction API to predict what game was streaming based on the color histogram of an image. The API worked pretty well and was fairly accurate. The main downside was that is was slow and inconsistent. Many
Detecting Games On Twitch.tv With Google Prediction API I've had this theory for a while now about a way to detect what game is currently being streamed. The basic idea is that a game has enough consistent image patterns that it should be possible to identify what it is in an automatic
Downloading Twitch.tv VODs Twitch currently doesn't give you a way to download VODs from their website. Just because they don't really let you, doesn't mean it's impossible. There are a few hoops you need to jump through in order to do so. When you watch a stream
Elastic Beanstalk Docker Best Practices In engineering at Mobivity, we have a common theme: "The less we have to manage, the better." The less we have to manage the better. We'd rather be focusing on product than infrastructure. Though with all these fancy managed services and PaaS
Data The Most Popular Messages In Twitch Chat If you’re familiar with Twitch TV, then you should be familiar with their chat. It’s entertaining to say the least, particularly in the very popular channels that have 5-50k viewers. Somebody will say something in chat that others end up copying and
