Data Analysis

The next step after data wrangling is usually data analysis, although sometimes you might jump straight to presentation if no particular analysis is required. The main objective of data analysis is to let the data tell a story, which may involve formal hypothesis testing or estimating coefficients of models, but can be as simple as visualising the data in different ways to look for patterns and trends.

In fact, I find the latter to be increasingly helpful as a complement to formal statistical analysis. The challenge is visualising complex datasets that have many dimensions. Since at best we can only look across two or three dimensions of a dataset at once, we have to be careful about the way we choose to ‘cut’ the data. Excel pivot tables and pivot charts are a pretty good way of doing this, although the presentation of the data in Excel is not as clean as it could be.

I also find it very helpful to think about the possible relationships that could exist in a dataset before diving in to the analysis. This does not necessarily involve coming up with formal hypotheses to test, but rather just thinking about what are the sensible correlations or patterns to look for.