Awesome

Technology Stack

The software and systems we use to build the dwyl platform.

• Technology Stack
• Why?
• What?
  • The PETAL Stack
    • Phoenix
      • Phoenix the "Most Loved" Framework in 2022
    • Elixir
      • Elixir is #2 in 2022
    • Tailwind CSS
    • Alpine.js
    • LiveView
  • Beginner Tutorials?
    • Phoenix/Elixir:
      • Small Projects That Showcase Phoenix
      • Elixir Utilities
    • Flutter
    • Miscellaneous
    • Our MVP
  • Database?
    • We Prefer PostgreSQL
      • List of Organizations Using PostgreSQL
  • Operating System?
  • Continuous Integration
  • Deployment
    • Application Server
    • SSL/TLS Encryption
• tl;dr
  • Why Try a "New Stack"?
    • Why Try Something New When We're Already Good with the "Old"...?
  • Making Difficult Decisions
    • Most "Application Architects" will pick one of these 3 options:
    • Toast Knife Analogy
    • Focussing on Long-term Benefits
      • Further Reading on Long-term Thinking
    • Contextualising Technology Adoption (Mini History Lesson)
    • Does it Scale?!?
    • What About Full Stack JavaScript?
    • Alternative Databases?
  • Radical Simplicity
  • Other Tech/Tools?
  • How to Propose NEW Tech/Tools?

Why?

As a team of people using technology to make digital products, <br /> it's essential to be unambiguous about the stack/tools we use, <br /> so that everyone is clear what we all need to master.<br />

If anything is unclear or you have any questions please ask.

What?

This document + diagrams describe the full "PETAL" Technology Stack we use for dwyl products/projects.

Each element in our stack was carefully selected based on its individual merits. <br /> When assembled into a seamless "machine", the stack is unrivaled for developer productivity and world-class quality!

The PETAL Stack

"PETAL" is an acronym¹ for the following elements:

Phoenix

Phoenix is a Web Application Framework that does not compromise on speed, reliability or maintainability! Phoenix is the "successor" to the incredibly popular "Ruby-on-Rails" framework. Built from scratch by highly experienced engineers who worked on/with Rails. It solves all of the speed/socket/scaling/concurrency, issues people felt when building/using Rails apps. The list of benefits Phoenix has over (virtually every) other Web Frameworks is extensive.<br /> Please see: dwyl/learn-phoenix-framework#our-top-10-reasons-why-phoenix <br />

Phoenix the "Most Loved" Framework in 2022

Phoenix tops the list of "Most Loved" Frameworks on the 2022 StackOverflow Community Survey ❤️

https://survey.stackoverflow.co/2022/#section-most-loved-dreaded-and-wanted-web-frameworks-and-technologies

Elixir

Elixir is the functional programming language used by the Phoenix framework. Elixir is a beautiful language written from scratch to be friendly, concise and efficient. Yes, Elixir not as "mainstream" as JavaScript, Java, C# or PHP, but the adoption is growing rapidly and most importantly many experienced developers are gravitating towards and describing it as their "most wanted" Also a language's popularity has more to do with the intellectual inertia people/companies have because they allow existing (legacy) codebases to dictate future development; i.e. sunk cost bias. see: dwyl/learn-elixir#key-advantages

Elixir is #2 in 2022

Elixir is the 2^nd "Most Loved" programming language:

https://survey.stackoverflow.co/2022/#section-most-loved-dreaded-and-wanted-programming-scripting-and-markup-languages

This is a good measure of how much people enjoy working in the language. And as we all know people who enjoy their work are better at doing it!

Tailwind CSS

Tailwind is the most sane way of creating a beautiful web app UI that can easily be extended by a team of people without fear of one person's change "breaking" another feature. Unlike "traditional" CSS which - as it's name implies - encourages "cascading" of styles, Tailwind makes the style of each component specific and local to that component. see: dwyl/learn-tailwind

Alpine.js

Alpine.js is a lightweight library for enhancing interactions in a web application. It's declarative, responsive and easy to learn. Alphine.js plays well with LiveView for progressive enhancements. see: dwyl/learn-alpine.js

LiveView

LiveView is a radically simplified way of building realtime web apps with significantly less code.

Beginner Tutorials?

We have crafted a "Complete Beginner's Guide" for each element in the stack, so that we:

• Document our collective learning while we are building projects. <br /> (because as humans we forget fast unless we capture it immediately!)
• Share our knowledge with other people so we can
  • Help to train (potential) new team members as quickly/effectively as possible.
  • Collectively iterate on our knowledge and "level-up" as a team!
  • "Onboard" the client team (who may want/need) to support/maintain the codebase/project if/when we seamlessly "hand over".
  • Inform the wider community of both technical and non-technical people ("stake holders") who are generally interested in understanding the project.
  • Enlighten other teams/organisations/agencies/etc. we aren't in direct contact with that there is a "more fun" way of building software!
• Make everyone's life easier/better by having a "launch pad" for rapid learning!

We have written several beginner tutorials that span our technology stack and tools we actively use in development.

Phoenix/Elixir:

Here are a few of our learning repositories pertaining to Phoenix and Phoenix Liveview.

1. Learn Elixir: dwyl/learn-elixir
2. Learn Phoenix: dwyl/learn-phoenix-framework
3. Counter (Liveview): dwyl/phoenix-liveview-counter-tutorial
4. Todo List (Liveview): dwyl/phoenix-liveview-todo-list-tutorial
5. Stopwatch (Liveview): dwyl/phoenix-liveview-stopwatch
6. Chat: dwyl/phoenix-chat-example
7. Chat (Liveview): dwyl/phoenix-liveview-chat-example
8. Realtime cursor tracking (Liveview): dwyl/phoenix-liveview-realtime-cursor-tracking-tutorial
9. Papertrail and Phoenix: dwyl/phoenix-papertrail-demo
10. Flutter and Phoenix: dwyl/flutter-phoenix-channels-demo

Small Projects That Showcase Phoenix

We have a couple of "internal" (but Open Source) projects that use Phoenix and serve as a good showcase for the stack:

1. Labels: dwyl/labels
2. Hits: dwyl/hits

Elixir Utilities

1. Useful: dwyl/useful - utility library.
2. Content: dwyl/content - content negotiation.

Flutter

In this section you will find learning repositories where you can learn Flutter and how to use it with other technologies.

1. Learn Flutter: dwyl/learn-flutter
2. Learn Dart: dwyl/learn-dart
3. Supabase and Flutter: dwyl/supabase-flutter-demo
4. Counter: dwyl/flutter-counter-example
5. Stopwatch: dwyl/flutter-stopwatch-tutorial
6. Todo list: dwyl/flutter-todo-list-tutorial
7. Bloc: dwyl/flutter-bloc-tutorial

Miscellaneous

In this section, we will list a few repos that explain concepts and tools that we are actively using while developing our app.

1. Payment processing: dwyl/learn-payment-processing
2. API design: dwyl/learn-api-design
3. Test-driven development dwyl/learn-tdd
4. Insomnia API client: dwyl/learn-insomnia
5. Github Pages deployment: dwyl/learn-github-pages

Our MVP

We have built a fully working MVP version of our App! Check it out at dwyl/mvp!

Database?

The reason we do not specify our Database in the "PETAL" Acronym is that Phoenix allows us to use any Relational Database.

By abstracting the data layer using "Ecto" the application is "decoupled" from the database. <br /> This means that if a client asks us to deploy to MySQL or Microsoft SQL Server <br /> (e.g. because they already have in-house capability for maintaining one of these databases) <br /> we can easily accommodate that without re-writing any of the Phoenix app!

We Prefer PostgreSQL

<img width="500" alt="postgres logo" src="https://cloud.githubusercontent.com/assets/194400/22939394/6cc00918-f2d6-11e6-8400-77886d70b520.png"/>

Our "standard" (preference) @dwyl is for Postgres. see: dwyl/learn-postgresql <br /> Postgres is the most "mature" Open Source Relational Database. It's 100% Free (including all "advanced" features) and has been deployed and battle-tested in every environment from AWS to "Bare Metal" and Google Cloud to Microsoft Azure!

Many well-known/successful apps rely on Postgres as their main database. <br /> NOT that you should adopt a particular technology based on who else is using it, <br /> but it's good to know that plenty of teams are getting excellent results with Postgres!

List of Organizations Using PostgreSQL

We have used most of the "popular" Relational Databases. e.g: MySQL, Microsoft SQL Server, Oracle and Aurora, etc; all RDBMS have their pros/cons. <br /> The reason we like/use Postgres is because the community is superb. There is a great "bank" of answered questions on StackOverflow and new questions get answered fast.

Operating System?

A "traditional" LAMP stack includes the Linux Operating System in the name. <br /> The "PETAL" stack runs on any (desktop/server) Operating System <br /> and can be deployed to any "cloud" infrastructure provider. <br />

While we have a strong preference for Unix (e.g. FreeBSD) or Linux (e.g. Ubuntu or CentOS) we know that <br /> both Phoenix and Postgres run on almost any environment including Microsoft Windows Desktop & Server.

Continuous Integration

We are using GitHub actions for Continuous Integration / Continuous Deployment.

For an example of this, including automatic deployment to Fly.io see: .github/workflows/ci.yml

Deployment

We make a point of deploying our work as soon as there is something worth showing to the target audience of "end users" so that we can get feedback as early as possible.

Lately we have been using Fly.io for deploying our Apps. The experience is superb. ❤️

Application Server

The Phoenix Application Server is hosted on (a minimum of) Two Linux Servers. <br /> (often many more which send messages one another to distribute load as a cluster). <br /> The "cluster" is managed by Erlang's "Supervisor". The Erlang Supervisor is the "Gold Standard" in infrastructure management, having been used by Telecoms companies for over 20 years in production with some Telcos reporting 99.9999999% ("nine nines") of "up-time".

It's far more likely that the infrastructure provider (e.g. AWS/Azure) will have a fault in their network/datacenter than an Erlang server "crashing".

SSL/TLS Encryption

All communication is over secure/encrypted channel (by default at all times) <br /> to protect the data/privacy of people using the applications we make. <br />

We recommend using the "Let's Encrypt" service for SSL Certificates it's 100% Free (and provided by a Non-Profit foundation) <br /> to help you get started, we wrote a step-by-step setup guide for apps deployed to Heroku: SSL-certificate-step-by-step-setup-instructions.md

<br /> <br /> <hr />

tl;dr

There is no shortage of options available for Technology Stack! <br /> See: https://www.google.com/search?q=technology+stack&tbm=isch <br /> So, how did we arrive at the conclusion that PETAL was "the one" for us...? <br /> We already had a really good Node.js Stack which worked well for us and our clients. so . . .

Why Try a "New Stack"?

Why Try Something New When We're Already Good with the "Old"...?

Our reasoning for considering an alternative approach/stack for building web apps was fueled by our curiousity and "shoshin". <br />

"The important thing is not to stop questioning. <br /> Curiosity has its own reason for existence." <br /> ~ Albert Einstein <br />

In November 2016 we (once again) questioned our assumptions, re-examined and surveyed the "landscape" of "emerging trends" in web app development. We were pleasantly surprised delighted to see the amazing progress made by the people in the Elixir / Phoenix community! Please see: dwyl/learn-phoenix-framework#questions

Making Difficult Decisions

One of the most "difficult decisions" you will make in your "career" is which technologies and tools you will use to deliver the desired solution/benefit to the "end users".

Most people have the Tech/Tools decision made for them by the company/organisation/boss they work for (e.g: Java -> Spring, Ruby -> Rails or PHP -> WordPress or Laravel, etc.) This is because most companies already have an existing app in "production", which you have been hired to extend.

Occasionally you will get the chance to build an app from "scratch"
however most of the time someone else (the "Architect") will make the decision for what "stack" to use on your behalf, so you will still be stuck with someone else's choices. If you are incredibly lucky the "Architect(s)" will have done their homework: surveyed the latest industry trends and investigated the "new and promising" technologies e.g: Stack Overflow "Most Wanted" list.

Most "Application Architects" will pick one of these 3 options:

1. Go with what you (already) know, use existing stack with a minor variation because it's "easy to deploy" with the existing infrastructure and will not get questioned by the "Executives", DevOps team or "Compliance" department. This is the easy choice and nobody ever got "fired" for sticking with what they know "works".

2. Buy the whizz-bang all-in-one solution sold to them by the "Consultant" from "Big Vendor XYZ" (outsource the thinking to a sales person who last wrote code in the 90's ... seems like a great idea ... NOT!)

3. Be "Bold" and try "Popular Framework XYZ" and hire an external team to build the new magic app. Then attempt to "up-skill" the internal team to maintain the code written by the consultants.

None of these choices is optimal, all have different levels of risk/reward. The "hardest" choice to make is the one where you try something totally different. The reality is that very few people have the time/resources/mindset/inclination to take a step back and open their minds to the idea that there might be a "better tool" for the job than the one they are currently using.

Toast Knife Analogy

Imagine Want to Make Yourself Some Toast. <br /> The "user story" for this would be:

As a peckish person <br /> I want a slice of tasty toast <br /> So that I can have some crunch for my brunch!

( let's assume that you bought a loaf of bread from a baker <br /> to reduce the options for solutions for simplicity <br /> i.e. not baking from scratch! )

The "traditional" way to "solve" the challenge of making toast: <br />

1. Cut bread with bread knife
2. Put sliced bread in toaster
3. Turn on toaster for pre-determined amount of time
4. Wait patiently for toaster to make toast

But ... what if instead the "old" way we just described, someone invented a way to toast the bread while slicing it...?!

Simply by using the "New Tool" for the job - in this case the "Toast Knife" - you can simplify the process to a single step! <br /> This is the power of being open to considering "New" Tools/Technologies! <br /> Get the same result in fewer than half the "steps"!

Focussing on Long-term Benefits

The short-term cost of learning a new stack (programming language or framework) is time, We contend that the 1 week learning time (depending on the focus of learners) will pay for itself within 1 month (often sooner if the team is large/distributed because the structure offered by Phoenix means everyone working on the project has a disciplined approach to adding new features)

Further Reading on Long-term Thinking

• https://hbr.org/2012/08/thinking-long-term-in-a-short
• https://hbr.org/2011/03/capitalism-for-the-long-term

Contextualising Technology Adoption (Mini History Lesson)

In 1996 Nokia introduced the "Communicator" and was a incredible revolutionary innovation! Internet, Email and Fax in your Pocket!! <br />

<br />

Nokia continued to dominate the mobile phone industry/market for the next decade producing the best-selling 5110 and 3310 some of us are old enough to remember! <br /> But by being "ahead" Nokia was unable to see the "contender" coming "out of nowhere" to challenge their position.

In 2006 nobody was making/buying "smart" mobile phones with glass touch screens that ran "apps" ... <br /> in January 2007 Steve Jobs introduced the iPhone and literally changed the industry!

The dominant/incumbent mobile phone maker Nokia had 49% market share in 2007 mocked Apple's lack of features, poor battery life and high price. <br /> By 2013 Nokia had 3% Market Share (for new device sales) and was sold off "for parts" to Microsoft while Apple was the most valuable company in history!

Many people still buy "feature phones" (the polite name for a device that does not have any "smart" functionality!) <br /> but few people can convincingly argue that the reason they don't have a smart phone is because they don't want one; <br /> over 90% 16-24 year olds own a smart phone and that number is expected to reach 100% by 2025 ... <br /> ask someone in their 20's if they would go "back" to using a non-smart phone and see what they say ... "No Way!!" <br /> they laugh uncomfortably and admit that: "My Smartphone is my Life!" <br /> We feel exactly the same about "old technology". Sure the "old way still works", but if you can inexpensively switch <br /> to something demonstrably better in every aspect, why would you stick with the "feature phone" of web frameworks...? <br /> It's like taking the Bus to work when there's a perfectly good teleporter right next to the bus stop!! Madness.

We are not suggesting that everyone is going to suddenly flock to the "PETAL" stack the way people adopted smart phones. This is merely an illustration that when a technology has a specific/measurable advantage to it's users the adoption can be fast.

In the case of programming languages or application frameworks, moving one framework to another is a much more difficult decision.

But one thing is for sure we are going to use the "smart phone" even if other people insist on using the "brick".

Does it Scale?!?

If you are new to web development, please focus on UX and forget about "scale"! <br />

Unless you work somewhere that already has "millions of users" and <br /> your team cannot consider anything that does not support a million concurrent connections...! <br />

But let's face it, most people have imaginary scaling issues not real ones. <br /> discussing "scalability" before you have 10,000 paying customers is a waste of time!! <br />

Stop worrying about "scalability" and instead focus on building something useful <br /> focus on User Experience not ("backend") scalability!

The good news is that Phoenix "scales" really well! <br /> see: phoenixframework.org/blog/the-road-to-2-million-websocket-connections

Forget about "scaling" until you have made something people want and are paying for! <br /> Then use the pile of cash you got from your product to hire "engineers" to make it available to more people!!

<br /> <br />

What About Full Stack JavaScript?

We still think that "Full Stack JavaScript" is a compelling proposition especially for people who are just starting out! It allows you to write one programming language on both the client and server; we get it! However we have learned from years of experience that it requires a lot more code and maintenance than PETAL for an inferior result in terms of UX/performance and developer productivity.

Alternative Databases?

If we were to consider an alternative to SQL, we would use RethinkDB: https://rethinkdb.com <br /> But we are relieved that the Phoenix team is focussed on PostgreSQL because that eliminates <br /> the "ambiguity" or "discussion" of "which database" to use! Postgres is a fantastic "general purpose" <br /> store that has a rich ("structured") query language that lets you JOIN data!! <br /> Also, now that Citus DB is Open Source we know that Postgres can easily handle billions of writes per day!

Radical Simplicity

“If it takes an hour to figure out what’s going on, well,<br /> that’s an hour that wasn’t spent doing something else more useful and interesting." <br /> ~ Rachel Kroll

Please read: https://www.radicalsimpli.city <br /> In the site/manifesto Stephan makes the case that Apps in 2021 have gotten far too complex:

He advocates for a return to basics:

No need for microservices message queues or other complex tech that is only relevant to 0.1% of mega scale companies:

We agree.

If by some luck our product reaches the point where we need mega scale (millions of people creating billions of items) we know that our chosen stack will scale just fine.

Before then the complexity will only slow us down and reduce the chances of success.

Other Tech/Tools?

We have written about our choice of programming language extensively in: learn-elixir/issues/102.

Our use of Elixir is for a very specific reason: we are building fault-tolerant realtime systems. For the type of App we are building, Erlang/OTP is the undisputed king on the server side. We could use almost any other language/framework, but it would be a lot more work for an inferior result.

If we need to build a specific feature requested by a person using our product, then we will 100% consider a technology that enables us to deliver it.

How to Propose NEW Tech/Tools?

The way to propose a specific tech/tool is simple: open an issue describe how the tech/tool will help us build a specific feature that has been requested by a person using our product.

Proactively create a new repo in the dwyl org to capture your own learning of the tech/tool you are proposing. e.g: dwyl?q=learn

Once you have invested the time to learn the tech/tool beyond "hello world" and are confident that it will help us achieve a specific end-goal, then please make the case for it.