Home

Awesome

rt-csp

roland's fun CSP for lithium repo

24April2017 repeat test from 22March2017

cp  ~/Downloads/14april2017-mozilla.prod-csp-sanitized-report.csv . # 1
tr -d '\r' < 14april2017-mozilla.prod-csp-sanitized-report.csv \ # 2
> unix-line-endings-14april2017-mozilla.prod-csp-sanitized-report.csv
./print-domain.rb  unix-line-endings-14april2017-mozilla.prod-csp-sanitized-report.csv \ #3
2>14april2017-stderr-mozilla-domains.txt >14april2017-stdout-mozilla-domains.txt
cat 14april2017-stdout-mozilla-domains.txt | sort | \ #4
uniq > 14april2017-unique-mozilla-domains.txt
grep FIELD3 14april2017-stderr-mozilla-domains.txt | sort \ #5
| uniq > 14april2017-stderr-non-http-non-https-field2.txt
grep URI 14april2017-stderr-mozilla-domains.txt | sort | uniq #6
/Users/rtanglao/.rbenv/versions/2.3.0/lib/ruby/2.3.0/uri/rfc3986_parser.rb:67:in `split': bad URI(is not URI?): http://support.mozilla.org/skins/2360640/fonts/bootstrap/glyphicons-halflings-regular%woff (URI::InvalidURIError)
PublicSuffix::DomainNotAllowed^^^ URI:nikkomsgchannel
PublicSuffix::DomainNotAllowed^^^ URI:s3.amazonaws.com
PublicSuffix::DomainNotAllowed^^^ URI:s3.eu-central-1.amazonaws.com

cp mozilla-good-bad-domains.md 14april2017-mozilla-good-bad-domains.md

22March2017

require 'rubygems'
require 'ccsv'

Ccsv.foreach(file) do |values|
  puts values[2] # values[2] to get the URI
end

uri = URI.parse("https://support.mozilla.org/t5/user/viewprofilepage/user-id/873432")
=> #<URI::HTTPS https://support.mozilla.org/t5/user/viewprofilepage/user-id/873432>
irb(main):006:0> domain = PublicSuffix.parse(uri.host)
domain.domain
=> "mozilla.org"

tr -d '\r' < mozilla.prod-csp-sanitized-report.csv \
> unix-line-endings-mozilla.prod-csp-sanitized-report.csv

./print-domain.rb  unix-line-endings-mozilla.prod-csp-sanitized-report.csv \
2>stderr-mozilla-domains.txt >stdout-mozilla-domains.txt

cat stdout-mozilla-domains.txt | sort | \
uniq > unique-mozilla-domains.txt

grep FIELD3 stderr-mozilla-domains.txt |sort | \
uniq > stderr-non-http-non-https-field2.txt

rtanglao13483:rt-csp rtanglao$ grep URI stderr-mozilla-domains.txt 
PublicSuffix::DomainNotAllowed^^^ URI:s3.amazonaws.com
PublicSuffix::DomainNotAllowed^^^ URI:s3.amazonaws.com

cat unique-mozilla-domains.txt stderr-non-http-non-https-field2.txt > mozilla-good-bad-domains.md
echo "http://s3.amazonaws.com" >> mozilla-good-bad-domains.md

20March2017

working on case 00134461 which is referenced in CSP bug 1339940 as well as HSTS bug 1340056

grep -v "[0-9a-z]*.addthis.com[0-9a-z]*,,,," mozilla.prod-csp-sanitized-report.csv \
> addthis.com-removed-mozilla.prod-csp-sanitized-report.csv

grep -v "[0-9a-z\/:]*.support.mozilla.org[-A-Z0-9a-z\/]*,,,," \
addthis.com-removed-mozilla.prod-csp-sanitized-report.csv > \
support.mozilla.org-removed-mozilla.prod-csp-sanitized-report.csv

grep -v "[0-9a-z\/:]*.youtube.com[-A-Z0-9a-z_=\/\&\?]*,,,," \
support.mozilla.org-removed-mozilla.prod-csp-sanitized-report.csv > \
youtube.com-removed-mozilla.prod-csp-sanitized-report.csv

grep -v "[0-9a-z\/:]*.addthisedge.com[-A-Z0-9a-z_=\/\&\?]*,,,," \
youtube.com-removed-mozilla.prod-csp-sanitized-report.csv > \
addthisedge.com-removed-mozilla.prod-csp-sanitized-report.csv

ggrep -Pv ",,[a-z\:\/\.]*support\.mozilla\.org[A-Z0-9a-z_=\/\&\?\-\%\.]*,,,," \
addthisedge.com-removed-mozilla.prod-csp-sanitized-report.csv > \
unicodesupport.mozilla.org-removed-mozilla.prod-csp-sanitized-report.csv

ggrep -Pv ",,[0-9a-z\:\/\.]*addthis\.com[A-Z0-9a-z_=\/\&\?\-\%\.]*,,,," \
unicodesupport.mozilla.org-removed-mozilla.prod-csp-sanitized-report.csv > \
reallyaddthis.com-removed-mozilla.prod-csp-sanitized-report.csv

ggrep -Pv ",,[-0-9a-z\:\/\.]*mxpnl\.net[A-Z0-9a-z_=\/\&\?\-\%\.]*,,,," \
reallyaddthis.com-removed-mozilla.prod-csp-sanitized-report.csv > \
mxpnl.net-removed-mozilla.prod-csp-sanitized-report.csv

ggrep -Pv ",,[-0-9a-z\:\/\.]*addthisedge\.com[A-Z0-9a-z_=\/\&\?\-\%\.]*,,,," \
mxpnl.net-removed-mozilla.prod-csp-sanitized-report.csv > \
reallyaddthisedge.com-removed-mozilla.prod-csp-sanitized-report.csv

grep -v vpaid.js \
reallyaddthisedge.com-removed-mozilla.prod-csp-sanitized-report.csv >\
vpaid.js-removed-mozilla.prod-csp-sanitized-report.csv

grep addthis -v \
vpaid.js-removed-mozilla.prod-csp-sanitized-report.csv > 
\really-really-addthis.com-removed-mozilla.prod-csp-sanitized-report.csv

grep -v stickyads \
really-really-addthis.com-removed-mozilla.prod-csp-sanitized-report.csv >\
stickyadstv.com-removed-mozilla.prod-csp-sanitized-report.csv