The Timeline Project

I have been fascinated by the depths of information we collect today about every event. Gangnam style hitting 1 billion views was a phenomenon and social networks contribute greatly to the viral “quotient” of things today. The YouTube Rewind page has evolved almost as a time capsule where pause, rewind and remember the videos that defined YouTube in that year. The way we remember time has to go beyond a tabular description of events in text along with dates. It has to be more meaningful, more illustrative and more relatable for everyone. That was the inspiration behind me kicking off this project.

All the code for this project is hosted on github here – https://github.com/vivek1729/Timeline

My thought process:

 What

I wanted to create a virtual time machine (No VR Gimmicks yet), may be just a plain old web app. I wanted users to experience what would it have been like in 1939 Germany when the second world war broke out, or how was 11th September 2001; what defined that day and how do we remember it now. I wanted to break down time to a month level and not more granular because frankly, as we go back information of happenings on a particular day of the month get more sparse.  So which pictures or videos defined September 2011 for Earth? What major events happened during that month? What were the general preferences of people then? Now, the last question is a bit tricky. It’s difficult to define the spirit of times. I started with Music. I figured why not look at what kind of music was popular in that era.

 

Data Source

The first and the obvious place I looked at to get comprehensive information about historic data was wikipedia. It was a pleasant surprise to have discovered year wise wikis like https://en.wikipedia.org/wiki/2016#May. Clearly the structure of the URL allows me to “ideally” look up any month and any year in a standardized format and look at the information wikipedia has about it. Unfortunately! Wikipedia does NOT have an official API to get structured information on these topics. So I had to go the old school way and scrape the page to get relevant information (it was a pain in the rear :P). The information I scraped was this (for eg: May 2016) :

 

May 2016 Wiki

 

Images

I grappled with problems here. There are still issues I face here. I decided to go with Google Image search to look up images pertaining to the events in the wiki page. I did not perform any natural language processing because the events description was terse and extracting key words might have caused some loss in context and inaccurate representation. The pain in the rear for the image search was again Google’s hard limit of I think a 100 API requests in a day! 🙁

 

Music

Billboards HOT 100 is the holy grail of popular music in an era and thankfully Wikipedia had me covered here too. The billboards website also maintains an archive that could come in handy.

 

Putting it all together in an API

I put this altogether in form of an API and made it available on github for anyone looking to extend or further develop a front end around this. The back end is a simple node server doing the scraping and parsing and presenting result in this format:

 

 

 Going forward

This is just a starting point for my project but I wanted to have a generic API that works for times as early as the late 1800’s. Social network trends can be a major source of information these days and we can always build that into the API. The final app I envisioned would have a slick UI where users would be presented with a tile of images/videos with text in bubbles and some music playing in the background which could make the user feel like a part of that era. I’ll try to make a prototype of the screen for the app and post it here in some time.

I would love to collaborate with anyone interested in taking this project further. Comment here or reach out to me at my contact details.

Leave a Reply

Your email address will not be published. Required fields are marked *