Home

Awesome

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests are very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products, Database of Databases, Google Open Source Glossary

Technology

Core Technology

Google InternalGoogle ExternalOpen Source / Real-World
MapReduceApache Hadoop, Spark
FlumeDataFlowApache Beam
Protocol BufferProtobuf, FlatBuffersCap'n Proto, Thrift, Avro, Amazon Ion, CBOR, kryo
StubbygRPCThrift, Bolt
ChubbyApache Zookeeper, etcd, HashiCorp Consul
Goops / PubSubCloud Pub/SubApache Kafka, Apache Pulsar, Facebook LogDevice
//baseabseil

Infrastructure

Google InternalGoogle ExternalOpen Source / Real-World
BorgKubernetesApache Mesos, Apache Aurora, HashiCorp Nomad
GSLBCloud Load Balancing - InternalAWS ELB, Istio, linkerd
OnePlatformAPI GatewaySwagger
GFE, Maglev, uberproxyCloud Load Balancing - HTTPS / Externalenvoy, AWS ALB, HAProxy, nginx, F5
uberproxy (sso proxy)Identity-Aware Proxybuzzfeed-sso, Pomerium
ZanzibarZanzibar Research PaperSpiceDB/authzed, Ory Keto, topaz, Opal, (iam)Keycloak, Warrant
data center hardwareopen compute
Jupiter, Starblaze
B4, Stargate, TE
USPS, Andromeda
ESDNFaucet
broccoli man

Storage

Google InternalGoogle ExternalOpen Source / Real-World
GFS/ColossusHDFS, Ceph, GlusterFS, MooseFS
BigTableCloud BigTablePrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB
SpannerCloud SpannerVitess, CockroachDB, TiDB
ColumnIO / CapacitorApache Parquet, ORC
sstablelevelDBRocksDB, pebble
zippySnappylz4
RecordIORiegeli, TFRecords, & in OR-Tools, szlstuffed-record-stream

Services

Google InternalGoogle ExternalOpen SourceSaaS
DremelBigQueryApache Drill, Presto, Spark(sort-of),AWS Athena, Redshift Spectrum, Snowflake
Dremel UIRedash, Metabase, Apache Superset
Search (Mustang, Alexandria)Elasticsearch, OpenSearch, Solr, Lucenealgolia
pubsubpubsubNATS.io, RabbitMQ, PubNubAWS SQS/SNS, AWS AppSync
MillWheelCloud DataflowApache Flink, beam
ColabColaboratoryJupyterObservable
PLXGoogle Data StudioMode
Monarchpaper
Napapaper
MakerSuiteMakerSuite

DevOps

Google InternalGoogle ExternalReal-World
AssimilatorDominator
BlazeBazelBuck, Pants, please.build, Blade, recc, BuildBuddy, flare.build
OncallPagerDuty, OpsGenie, VictorOps
varz/borgmon/monarchCloud MonitoringDatadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, bosun, also this and this
ViceroyCloud MonitoringGrafana
Exception/Error Tracking (thirdeye)Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps, ELK stack
styleguidesgoogle styleguidesPEP-8, HoundCI auto-style-reviewer
SpongeEngFlow
Blaze/Forge/TAP/BuildCopCloud Buildbuildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone
Forge/ObjFSEngFlow, flare.build
Sandman(test env)/Guitar
Sisyphus / RapidSpinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson
MPMDocker, OCI
borg / borgcfg / gclJsonnet, CueAWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, kubecfg, pulumi, Nix
logging, analogStackDriverlogstash, fluentd, PaperTrail, cernan, loki
CodeSearch, GrimoireZoekt kythe Code Search (for Google open source code only, with separate UI for Android and Chromium. Go CLI source.)Sourcegraph, OpenGrok, livegrep
Critique, Gerrit, Mondrian etc.GerritReviewable , Phabricator
ciderEclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab), DevZero
buganizerGoogle Issue TrackerJIRA, bugzilla, github issues, Linear
BugjugglerSnoozeThis
ToTTGoogle Test BlogIncrement
Copybara / MOECopybara, MOE
workflow/dependency managementLuigi, Airflow, digdag, Pachyderm, Dask
ErrorProneErrorProneSpotBugs, FindBugs
Dapperstackdriver tracezipkin, OpenTelemetry, jaeger, LightStep, Honeycomb
C++ Tips of the WeekAbseil C++ Tips of the Week
DiRTChaosMonkey, aws fis
Rosiemicroplane, silver-platter
API Improvements ProposalsAIP
g4 {fix, submit}Trunk.io, Graphite
proberscloudprober

Security

Google InternalGoogle ExternalOpen Source
prodaccess/LOASKeymaster
prod secrets/identity managementchamber, knox, SPIFFE

IT / Operations / Misc

Google InternalReal-World
software engineering at google 1, 2Software Engineering at Google: Lessons Learned from Programming Over Time
SRE @ google
valentineVault, 1Password, Lastpass, pass, keeper
OWNERS files in repogithub CODEOWNERS
snippetsKhan/snippets
SnipItrecordit, CloudApp, dropbox screenshots, Snippyly
stuff (SaaS IT management)productiv, intello, zylo
stuff (Device Management)jamf
device security monitoringRed Canary
beyondcorpbeyondcorp
go/ linksgolinks, go, Goat, trotto, go-shorten
google3 philosophyinnersource, monorepo, YouTube talk
doing code reviewcode review
safely sharing 1-time secretssendsecure.ly, croc, onetimesecret, privatebin
messagingmattermost, Slack, gchat
tech talksTechTalks @ Google
g3docwritethedocs, docs-as-code
GUTSspoke, freshservice
MOMAglean

PeopleOps / Culture

Google InternalReal-World
OKRCultureAmp OKR, Lattice Goals, Ally, workboard
HRIS/ERPNamely, BambooHR, Workday, Paylocity
peer bonusbonus.ly
kudosheytaco, slack ++ bot
perksfond
talksTalks @ Google
doryslido
edge, people ops trainingLifeLabs
googlegeistCulture Amp, humu, tinypulse, peakon
MengSearch Inside Yourself
LazloWork Rules
Claire StapletonTech Support - existential advice for the modern tech worker
books about googleHow Google Works, In The Plex, Software Engineering at Google

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies/products mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated.