I recently gave a talk at the local Bay Area Quantified Self Meeting which was a great experience with interesting people and awesome presentations.
I am going to talk about the what I have found is the importance of stepping out of your daily thing and periodically snapshotting your data to help you quantify your goals and discovery more about yourself.
Like many of you, I collect a lot of data which I represent some of it in this heat map. Even with what I affectionately call the “great hard drive crash of ’05”, I still have probably a few hundred thousand data points. And live on the premise that with enough data, answers present themselves.
The data generally comes from great services on the Internet that I use like Flickr, Foursquare, Flixster, and Fitbit. I use these services because a service like Foursquare has much better data about venues for example than I could create myself. The problem is that the data is not necessarily mine. Unfortunately, it lives with them and if they go out of business, I could lose all the data I gathered and it would be 2005 all over again.
Another problem that if I wanted to correlate data between services, I really don’t have that ability today. If I wanted to mash up my Foursquare checkins with movies I watched and rated, I could find out where and with who I see the best movies. So I decided to aggregate the data into a centralized database that I can control and I can do whatever I want with it. Most importantly, back it up.
To pull this data, I have a system to run scripts every 10 mins to pull data from the services with their APIs or RSS feeds where ever possible. Some things I have to or want to enter data in manually. Like health data when at the doctor’s office or record when I sick, I have a mechanism for that.
In the end, I have a ‘big data’ of my own events in my own database that I could use for all sorts of different things. I use this large data source for a couple of types of things. For what I call nostalgia, I do things like lifelogging and something I call ‘Audrey Answers‘. For quantification and discovery about myself, I use it for goal tracking, and of course annual reporting
Since I have all this data, I can display in interesting ways for example, I have relive the day I saw the Big Buddha outside of Hong Kong including the places I visited, pics I took, even the miles I walked.
I can also create dynamic single purpose websites to share. My dog for example has his own website at http://gizmato.com built from a simple Lifestream search of tags and keywords.
Also with this data, I can do what I call ‘Audrey Answers’ in which I can ask questions like what was my cholesterol last year, have I ever eaten at Gary Danko, did I like the movie Inception?
But ultimately, I have a goal driven person, so I mostly use this data for tracking goals and objectives.There is a difference between goals and objectives. Goals are lofty concepts like ‘being a better person’. The magic happens when you translate goals into actionable objectives, like to be healthy, drink more water. To be more well rounded and wise, visit a new place every year or do a new thing.
I can track these on a daily basis, like if I have gotten enough water in that day or if I walked enough steps but it is very myopic and suffers from the ‘trees among the forest’ problem. So I also have annual dashboards to show my progress for the year. Good for the year but not how it looks over longer stretches of time. But I stumbled across Nicholas Felton who creating personal annual reports since 2005. It resonated with me on how to snapshot my progress in life and how to see better trends over the time of my life.
My first report was for 2011 and I was just getting my feet wet. The first thing I realized that the act of creating a story to tell someone else my story made me analyze the data differently. For example, I knew now much soda I drank but when I translated to gallons per year, I was able to get more insight really how much that was. Also, I saw that I lost 87 pounds that year but gained 94 back. So the act of analysis gave new insights.
In 2012, I got more of my stride. This for example was a new visualization for me to surface chart my data sources to see how data I have and since when. I also started a narrative of my findings. This story telling allowed me to translate the data into a better representation of the kind of person I was that year. Like I saw a move about every other day, generally at home and marathoned one TV episode per day. I also did more trending over time. So I was able to see my income versus expenses for the past 5 years so give me a picture of how I was handling money. Or that bank charges I am paying is increasing every year.
In 2013, I tried some new visualizations like this Wordcloud of the words I used that year. For all the 13,000 words in my database for 2013, this showed me which words were used more. Since I believe that the words I use represents who am I or what I find important, then the Wordcloud was very insightful of that mentality. I also started a ‘typical week’ page to discover more about myself. So for a normal week in 2013 I was able to discover, I walked 32 miles, ate 5 bowls of cereal, drank 47 glasses of water, and had only an 8% chance of traveling.
All this is good insights and fun but what I really get out this data is tracking results against goals. Did I mention that I am goal driven? Case and point. I discovered that 64% of my 1800 or so tasks I completed last year was not contributing to my most important projects. Actually, not in projects at all. So everyday, I am doing crap and not what I established as important in my life.
So with this insight, I reset my objectives for this year to complete a set number of tasks per day and ½ of them need to be part of projects I deemed ‘high priority’. With this, I believe that I will fulfil those big aspirations.
So I think this is an important activity. I encourage you to take the time to pause, step out of yourself and reflect. Analyze your data at set time intervals is a great way. Measure that up to what you want out of life. In the end, you will thank yourself.
If you have other questions, here is my contact information and have more information of what I talked about and what I didn’t at leerogers.com.