Awesome
MovieRater
Crawl and merge ptt/imdb/yahoo movie data, help easy search high rating movie in Chinese and English.
Quick Start
nvm use
npm install -g yarn
docker compose up
yarn
yarn build && yarn start
then open http://localhost:3003
Get the latest Data locally
yarn setup
yarn mergedata
Developement
Please open three command line:
For UI developement, server run at http://localhost:3004:
docker compose up
yarn start
yarn webpack
For server developement, server run at http://localhost:3003:
docker compose up
yarn tsc:w
TZ=Asia/Taipei yarn nodemon
Test
yarn test
Debug Test in Vscode
yarn tsc:w
- Edit
Debug Test
section in ./.vscode/launch.json, for Example, if you want to debugnetflixCrawler.ts
file, modify theargs
as the following:
"name": "Debug Test",
....
"args": [
"dist/test/netflixCrawler.test.js",
"--no-timeouts"
],
- Set the breakpoint and click the
Debug Test
in VscodeRun and Debug
Application flow
- Server start
- Load data in cache, include recent movie list, all merged data
- Start scheduler for crawl yahoo/imdb/ptt
Crawler
- To manually run a single crawler you could reference firstTimeSetup.ts, and run
yarn setup
Reference
The project UI is using Material-UI
Deployment
Terraform
gcloud auth application-default login
cd terraform
terraform init
terraform plan
Heroku
It is automatically built and deploy on heroku.