I tracked my life for 4 years, here’s what I discovered about myself

TLDR: I run a correlation analysis between all of my daily self-tracked metrics. 60% of the statistically significant results are “obvious” relationships, while the rest are surprising, meaningful, and actionable.

My journey with self-tracking

I’ve been keeping a daily log of the day’s activities for approximately the past 4 years. I track my fitness, diet, sleep, mindfulness, mood, goals, good habits, bad habits, time spent, and personality traits.

For a full dive into how I started quantifying my life, the categories of data that I track, and the reasons why I do it, please check out my most recent post.

Why I analyze my quantified self data

I don’t collect data just for the sake of collecting data, trust me I have more interesting hobbies than that. I collect this data to discover meaningful relationships between the aspects of my life that I care about, and learn new things about myself that I can use for future decisions, both big and small.

Besides uncovering insights for future decision-making, it’s just fun. I’ve always been a data-driven person, and have enjoyed visualizing data and finding the hidden meaning in it. It’s one thing to experience that feeling on a data set, but a completely another one to have the data set be something that has such direct implications on your day-to-day.

Seriously, if you could answer questions like “What makes me happy?” or “When do I feel my most healthy?” in an objective and unbiased way, wouldn’t you want to do that?

The analysis that I do on my personal data

  • Stats: Basic computations like “Going to the gym 4 times a week on average”

I’m going to focus on correlations today, mostly because I find them fascinating & insightful, and they make some pretty charts that I’m excited to share with you.

Poke around my data

I have an interactive visualization on my data blog where you can see the analysis that I do on my data. Here’s a screenshot (since I can’t embed the whole thing here), but if you’re interested you can go check it out here.

Seriously, the data vis is pretty fun. Click here to go look at the analysis of about 30 health, fitness, diet, and mood data sources.

My self-analysis

The data in the charts above is probably much more fascinating to me than to you, as it helps tell the story of the past couple of years of my life.

With that in mind, I’ve spent a lot of time diving into all of these correlations. Here are the top conclusions that I’ve come around to realizing.

The metrics cluster up

I’ve found “families” of metrics that tend to move together and often fall in the same category of “good” or “bad” metrics. For example, days/weeks when I don’t eat a healthy diet, I’m much more likely to also be sleeping poorly, working out less, drinking more alcohol, and feeling worse about myself, all at the same time! Likewise, some of my good habits like doing yoga, meditating, flossing, eating healthy, are most likely to be recorded on the same day.

What can I say, I prefer to live life at either end of the extreme. It’s not as fun in the middle!

Jokes aside, that’s actually quite a valuable insight. Since recognizing this, I’ve gotten more serious about my morning routine, making sure I start the day off on the right foot.

Not always clear what causes what

Because of the above insight on metric clusters, I’m interested to see if there’s a particular metric that could be the trigger for a “good” or “bad” cluster of data for the day. My current analysis just looks for correlation, but not causation, so I can’t determine what those triggers might be… yet…

I’ll be diving into that in a future analysis. If you’re reading this and have some resources for me to look into to do a causal analysis, please reach out!

Obvious vs surprising relationships

I plugged in all of my data (77 metrics at the time) into my correlation notebook and did the correlation analysis on the daily timeframe. Even when looking at the last year of data, and filtering for a p-value of <= 0.05, I had 197 highly statistically significant relationships.

Wow, that’s a lot! I decided to cut it down to the top 50 from that set and analyze each one.

What I found was that some of them seemed “obvious” because they were very clearly related (like going to the gym and exercise time for the day, or time awake in the morning and total time asleep that night), but there is a fair share of them that surprised me.

30 of the top 50 relationships were directly linked to one another, while the other 20 of 50 relationships were insightful, and showed me relationships between my data that I wouldn’t have assumed without this type of analysis.

The insightful relationships

The “obvious” relationships wouldn’t be very fun to talk about, but here are the top 20 insightful relationships that I found:

  • When Anxiety is up, Confidence is down

Applications of correlation analysis

As you can see from the relationships above, there’s a mixed bag of insights.

Some are logical, some confirm the consequences of good or bad habits, some point to habits that are proven to make me feel a certain way. And some, they are just plain funny (don’t worry unnamed friend, we can still hang out even though you’re a bad influence on me. You know who you are).

For those that are into biohacking or experimentation, this kind of analysis can be quite revealing to see the effects of those experiments. One QS community member even cured himself of Crohn's disease by experimenting with various supplements, nutrition, and fitness regiments, and correlating those experiments with his symptoms.

What’s next for me

This analysis has truly been a ton of fun to do, and it’s inspired me to keep on digging into my data. Here’s my gameplan for the next couple of analyses and write-ups.

  • Integrated Data: This whole data set is manually tracked, but I’m going to be extracting some other interesting Fitness, Time, Music, and Productivity data to add to the mix.

If you enjoyed this post and are interested in future writings and online visualization tools, feel free to drop your email here and I’ll let you know when they are up.

Have data of your own that you would like to visualize?

If you have your own self-tracked data (in a CSV format) that you’d like to do similar correlation analysis for, you can do so on this site! Check out this data exploration notebook for a fun and insightful time!

Writings about the Quantified Self. https://dailyvis.com/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store