Lewis is a causality-based system that uses probabilistic contrastive counterfactuals for generating post-hoc explanations for black-box decision-making algorithms. First, it provides insights into what causes an algorithm's decisions at the global, local and contextual (sub-population) levels. The explanations generated by Lewis are described in terms of three novel probabilistic measures-- necessity score, sufficiency score and necessity and sufficiency score-- that quantify the influence of attributes toward an algorithm's decision. Furthermore, for individuals negatively impacted by the algorithm's decisions, Lewis generates actionable recourse translatable into real-world interventions.