Home

Awesome

Twitter Data Toolkit

Twitter Data Toolkit is a Node.js project that allows users to collect Tweets using Twitter's new v2 endpoints and aggregate Tweets into four JSON files (Tweets, Extended Tweets, Users and Places) without having to interact with code. Everything's done through a simple web application. Feel free to email me (s.kishore@auckland.ac.nz) if you need more information.

Screenshot

Reference

If you use this software, please reference: Kishore, S., Peko, G., & Sundaram, D. (2019). Looking Through the Twitter Glass: Bridging the Data – Researcher Gap. Paper presented at the Americas Conference on Information Systems (AMCIS) 2019, Cancun, Mexico.

Instructions for Academics (Windows/Linux/macOS)

  1. Apply for access to the Academic Research product track (https://developer.twitter.com/en/solutions/academic-research/products-for-researchers).
  2. On approval, create an app under "Apps" (https://developer.twitter.com/en/docs/projects/overview).
  3. Download and install Node.js (https://nodejs.org/en/download/).
  4. If you have a GitHub account, clone or fork this project. If you don't have a GitHub account, click the green "Clone or download" button on the top-right and download the ZIP file.
  5. From the command line, change directory into "twitter-data-toolkit", type "npm install" to install dependencies, and "npm start" to start the application.
  6. Finally, in a web browser, type in "https://localhost:8080" to access the web application and start collecting data! 🎉

FAQ

How is this different to other projects?

This project focuses on those with a non-technical background. When you simply want to collect some data to explore an idea, this is a great place to start.

Why should I use the Twitter API?

Twitter recently introduced the Academic Research product track. The API's available through this track are particularly useful in a research context as they allow you to collect full-fidelity data that others can replicate. Note that the Academic Research product track is currently closed. It now costs $100 USD per 10,000-50,000 tweets.

How much does it cost to use the Twitter API?

Nothing. Once a research project is approved, user's can collect up to 10,000,000 Tweets per month. Note that the Academic Research product track is currently closed. It now costs $100 USD per 10,000-50,000 tweets.