Awesome
This repository contains code to build cosponsorship networks from bills (and motions) passed in the Finnish Parliament.
For related work, see the finpar
package.
HOWTO
Important note: the scraper is currently broken due to a website redesign that makes the search engine inaccessible for various projects, including this one. It is unlikely that this issue will get fixed in the near future, but please feel free to ask for the raw data collected for years 1999–2014 (the code will process it without breaking).
DATA
Bills
url
-- bill URLyear
-- year of introductionauthors
-- semicolon-separated sponsor namesn_au
-- total number of sponsorslegislature
-- legislature number (35 or 36)
Sponsors
profile_url
-- URL to access the sponsor profileurl
-- alternative profile URL (see below)year
-- year of entry in parliamentphoto_url
-- photo URLname
-- sponsor namename_full
-- sponsor full name (same as in bills data)born
-- year of birthparty
-- political party, abbreviatedconstituency
-- constituency, stored as the string to its Wikipedia Suomi entrypartyname
-- political party, full nameparty_length
-- number of political parties the sponsor has belonged tomandate
-- semicolon-separated years of mandate, used to compute thenyears
seniority variablesex
-- gender (F/M), imputed from first namesphoto
-- photo URL, simplified to its filename
The Eduskunta website uses a very strange URL system: two URLs are necessary to scrape the sponsors, which explains the four different URL variables (two for the sponsor profile, one for the original photo URL and one for the shortened version).
If the URL at line 132 in data.r
fails to work (which is almost certainly going to be the case):
- visit this index page
- extract the string stored into the
THWIDS
parameter when clicking 'Next page' (Seuraava at the bottom-right of the page; copy the URL to the clipboard to view it) - remove the part before the first period, which should be
100
- replace the string
.21/1421346561_16245
on line 110 with the new string
THANKS
Thanks to Leo Lahti, Joona Lehtomäki and Juha Yrjölä for pointers on how to impute gender from Finnish names: see this discussion and this discussion.