In recent years, legal scholars have become acutely concerned with the hedonic consequences of incarceration. Despite this interest, no research has simultaneously tested (1) whether current incarceration and recent incarceration lead to declines in happiness, and (2) whether the direct effects of imprisonment (what Gresham Sykes referred to as the "pains of imprisonment') or the indirect effects of imprisonment (what scholars have come to call the "collateral consequences" of imprisonment) explain these effects, although there are compelling reasons to expect both. In this Article, we consider research on the causes of happiness and the consequences of incarceration to generate three hypotheses: the pains of imprisonment hypothesis, the incomplete adaptation hypothesis, and the selection hypothesis. We then use data from the Fragile Families and Child Wellbeing Study and an array of rigorous statistical methods to isolate effects of current incarceration and recent incarceration on happiness.The results suggest that current incarceration has hedonic consequences, leading to statistically significant reductions in happiness across a range of statistical models. Indeed, current incarceration is the only factor more strongly correlated with current happiness than prior happiness. Furthermore, the indirect effects of imprisonment do little to explain these hedonic consequences, providing support for the pains of imprisonment hypothesis. The only inmates whose happiness does not follow this pattern were affectively unusual. They either disproportionately suffered from depression before incarceration or were profoundly unhappy prior to their incarceration (suggesting they had little possibility of becoming less happy while incarcerated). Recent incarceration, on the other hand, has no discernible effects on happiness after accounting for existing individual personality traits. Taken together, these results suggest that in terms of happiness lost, it is possible to match punishments with crimes.