One annoying error which I often see when running Hadoop jobs is this:

java.lang.OutOfMemoryError: GC overhead limit exceeded

The cause of this error is that Java is spending a lot of time inside the garbage collector, and is not freeing up large chunks of memory. When this error …

In a recent blog post, Panos Ipeirotis asks the following question (loosely paraphrased).

Consider a set of n oracles or mechanical turks, each with a probability q of returning an erroneous response. The capacity of a channel with error probability q is C(q,n), and information theoretic channel capacity …

Great news! A murder victim has been found. No slow news day today! The story is already written, now a title needs to be selected. The clever reporter who wrote the story has come up with two potential titles - "Murder victim found in adult entertainment venue" and "Headless Body found …

Python is not ready for the big leagues, at least if you need to deal with concurrency. I know this is a fairly controversial statement, and I plan to back it up with very specific critiques. The general gist of my critique is that python fails to properly utilize modern …

At their most basic, monads are a monoid in the category of endofunctors. But that's an explanation that only appears to mathematicians. They are also a design pattern, but that's an explanation that only appeals to computer geeks.

You've read many monad tutorials. For instance, monads are like monsters. No …

One of the wonderful features of Scala and other high level languages is that they are very expressive. Very often, one can represent business objects as a simple map, e.g.:

val headers = Map("User-Agent" -> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0 …

So I've just launched my new startup, BeerBnB. It's a hip little site matching beer drinkers with specialty microbreweries - AirBnB for drinkers, or maybe eBay for brewers. My marketer growth hacker has gotten some early publicity by advertising in the bathroom of a few bars - the result was 794 unique …

In Bayesian reasoning, the fundamental problem is the following. Given a prior distribution \(@p(x)\)@, and some set of evidence \(@E\)@, compute a posterior distribution on \(@x\)@ namely \(@p(x | E)\)@. For example, \(@x\)@ might be the conversion rate of some email. Before you have any evidence you might expect …

Hypst is an early stage startup - the elevator pitch is "like Facebook, before it became mainstream". The founders of Hypst have read a lot about A/B testing and statistics, and they decide to use the techniques they learned to improve engagement.

The first thing they notice is that not …

By now, it is a fairly uncontroversial opinion that ORMs create a large number of difficulties when developing larger systems. The have been famously called the Vietnam of Computer Science. The main alternative to ORMs is manually constructing SQL by hand, but unfortunately that is a rather dangerous thing to …