In a recent announcement, Twitter has shared how its has achieved its long-standing goal of allowing people to search through every Tweet every published. We all have had moments when we have shared latest happenings or tweeted about weird feelings. Now thanks to this recent update which will be rolled out in next few days, it is all searchable.
Twitter had to scale its systems, and using its already real-time index it has managed to not only sort old posts since 2006 but also those which were published during the up-gradation process.
The full index project was led by Yi Zhuang and Paul Burstein. The full index is a major infrastructure investment and part of ongoing improvements to the search and discovery experience on Twitter. “We built a search service that efficiently indexes roughly half a trillion documents and serves queries with an average latency of under 100ms,” Twitter’s Yi Zhuang explains in a blog post.