The Adventure-IT tech stack

This blog entry will focus on giving a high level view of the tech stack used to create Adventure-IT. It might not be for everyone, but since learning new tech is a big part of why Adventure-IT exists, I thought it’ll be nice to write a bit about the tech.

<rant>

Full stack Development

The Full stack developer myth

There was a time that I would’ve argued that full stack developers are about as real as unicorns. Then I found out that Scotland’s national animal is a unicorn, and it got me thinking, what’s it like to ride a unicorn? But also, if unicorns are real then maybe, there might be something like close to a full stack developer out there.

“Preposterous!” I hear you think. Please, allow me the opportunity to rant on how a Full stack developer might just be a real boy.

A jack of all trades is a master of none, but oftentimes better than a master of one

To tackle this debate, I’m going to divide the Software Engineer term into 3 main categories:

• DevOps
• Front end
• Back end

Now, the main argument I hear (mostly from past me) is that it’s impossible to be an expert in more than one of those. You have to pick one, and make it yours. This will then become what you’re known for, and it will be your bread and butter. For example, if you go onto google to search for jobs, you’ll probably start by typing in your category. And I have to admit, this is a very viable option to take, there’s nothing wrong with it, being an expert is something to be proud of!

But what if I don’t want to be an expert? Well, this is where the mythical full stack developer comes in.

“Ah, so fullstack is for people that aren’t good enough to become an expert!” I hear you think again. Nope, I feel like this is a common fallacy. To explain, consider the following analogy of the electromagnetic spectrum.

For argument’s sake, lets divide the spectrum into the following 3 categories:

• Short wavelength: Ultra-violet, x-rays, gamma rays.
• Medium wavelength: Purple to red light (to humans)
• Long wavelength Infrared, Microwaves, Radio waves

Us mortal humans only see the medium wavelength part of the electromagnetic spectrum (shocking, I know). We can almost say that we have become exports in this category. What if we can adjust our section of what we can see by sliding our view to the left or right of the spectrum? So instead of being able to see Purple to red, what if we chose to rather be able to see the wavelengths from x-rays to only lets say the colour green. Now, we no longer have 3 categories, but rather a view to only a select section of the entire electromagnetic spectrum, and we are able to slide the view from one end to the other. It’s important to note that our view doesn’t grow or shrink in size, the size stays constant, but we have the ability to slide our view around to see the parts we’re interested in.

Where was I going with this?

In the same way, lets change the 3 categories of a software engineer to rather be a spectrum. Where knowing less of one thing doesn’t result in knowing less overall, but rather that we know more about something else in its place. For example, it can be perfectly possible for someone to know 33% of Front end, 33% back end and 33% of DevOps. Or change the values around however you desire. We can slide our view around, just like we did to see other wavelengths in the electromagnetic spectrum, without having to increase or decrease our view’s overall size.

Cool, I somehow made my analogy work, I think, honestly I don’t know where I’m going with this rant. Words mean nothing to me at this point

This gives me a nice definition of expert vs full stack. An expert is someone with lets say 80% of knowledge in one field, but only 20% of knowledge in the other 2 combined. Whereas a full stack developer tries to balance out the values to be equal for all the categories.

Now I hear you think (again) “What type of developer is best?”. To which I say, none, or actually both? Full stack developers will always need the experts to help with the really tricky parts of each category, and experts will always need others to do the parts that they know nothing of.

Why not just have an expert for each category? I hear you think, again. Well, you can, but there are arguments against this, just like there are arguments against only hiring full stack developers. Experts might be more expensive or work slower because they get stuck in finer technical details which aren’t always relevant (because they enjoy the details). Whereas full stack developers might be work faster, because they typically value full stack completion over technical perfection.

With all this said and done, lets just agree that both exist and are equally important (what a sell out, taking the easy way out. I hear you think, again)

</rant>

Okay I’m done with that pointless rant now. Sorry about that. Moving on

The actual tech stack of Adventure-IT

Now, in order to build this app into existence, I needed to become a full stack developer. In order to do this, I had the following plan for each category:

(“I thought there were no categories but rather a spectrum”… We’re moving on!)

• Front end: This was my most proficient category since it was my day job for a couple of years. Therefore, yolo.
• Back end: At this point in time I recently started to spend most of my free time doing online courses and getting myself up to speed with everything back end related. Therefore, yolo.
• Dev Ops: I’m f#cked :( wait I have a friend that is an expert in the field! (10%er for the win)! Therefore, yolo.

In short, yolo is the plan of action. Learn by doing! Carpe diem! - How millennial of me, I know

The Front End

To pick a front end seems like a daunting task. As a software engineer, programming languages or frameworks aren’t really a big deal for me. They are all very similar in features and concepts. To be honest, you’ll be fine with picking either one of several options like Angular, React or Flutter.

Key considerations to keep in mind however is which platform your front end should support. I really want to be able to expand onto all platforms, be it Android, Apple or Web. For this you could either do a web app in Angular or React and use something like Progressive web apps to get it into a mobile device such as Android or Apple. This is nice and all, but better options exists, such as React Native, Flutter or maybe even Kotlin Multiplatform.

In the end, I decided to go with Flutter. Mostly because at this point it was what I was most familiar with. Over the last couple of years, I’ve mentored 2 student projects where they used Flutter (it’s how I was introduced to the framework). I also worked at a Fintech startup that used Flutter as their front end, and through all this, I haven’t really found a reason to not use Flutter.

Another very nice thing about Flutter is that I found it very easy to find examples on features I’d want to use. For example, the official Flutter YouTube playlist is an excellent source of info, and I’d recommend it to any Flutter dev.

Was this a good decision?

Yes, I do not regret going with flutter at all thus far. I wish I knew a little but more about the best practices before I jumped into development though. Mostly because I didn’t know about state management libraries such as BLoC. So I created my own worse version of it using the standard Providers and Change Notifiers. Even though I’m happy with my implementation, I do think it would’ve been way better to just go with BLoC. Live and learn I guess. At some point I’ll have to revisit and refactor this though (Tech debt printer goes brrrrrr)

Important Update:

The lack of true native components are kind of a deal-breaker for me (I feel so lied to)

I’ve recently been made aware of some of Flutter's dark secrets in the following video,

and to be honest this kind of spoiled Flutter for me. A lot of what was said in the video rang true with my experience with flutter, and I’ve suddenly realised that the problems that I’ve been having wasn’t purely because of my lack of knowledge in flutter.

Moving forward I shall be looking more into React Native or maybe even Kotlin Multiplatform to see how they compare. One negative of React Native for me is the JavaScript environment coupled with what seems to be XML views. Granted, I have exactly 0% experience in React Native so this is probably a non issue, but from an outside perspective this is enough for me to first want to try out Kotlin Multiplatform.

The Back End

Honestly, I don’t have much to say here about which back end tech stack to go with. Spring Boot seemed like the framework of choice. I already liked Java, and was eager to learn more Kotlin. So I decided to go with Kotlin, Gradle build tool along with it. As for the database layer, I picked Postgres, since I couldn’t find a single reason why I’d pick anything else.

Something I did debate for a bit was, should I go full reactive or not? It meant that I would drop Hibernate and rather use R2DBC. This scared me a bit, since R2DBC was quite new and I don’t think it was even production ready when I started. But it meant that I can drop the very temperamental Hibernate, so it ended up being a bit of a no-brainer to be honest.

Initially I started out with the normal reactive stack, but pretty soon I was stuck in Mono/Flux nested hell. That many nested code blocks was not fun at all.
Luckily, I was working in Kotlin, and my expert friend (10%er) showed me the magic of coroutines. Suddenly I can write suspend functions that didn’t need any nesting at all, this was a game changer. The code was so much more clear and concise, easy to read and understand. 100% would recommend this to anyone.

R2DBC also suddenly came alive with the CoroutineCrudRepository. Such light weight, much wow. Glad I’m done with hibernate! The dreaded n+1 problems where so much easier to find and get rid of!

For handling authentication and authorisation I did not want to do any custom implementations. Security is a complicated topic, and I do not have enough experience in this area to try and do my own thing. Luckily I already ran into this topic before and found that Keycloak seems to be a very solid open source solution.

Not only does it handle all the best practices for handling auths, but it also helps me integrate with other services like Firebase auth or Login with Google using OpenID Connect(OIDC). The learning curve of Keycloak is a bit high, but once everything is up and running it does seem to be very worth it.

Architecture wise I decided to go with a microservice approach. This did complicate the architecture a lot, but as I’m in it for the tech, I decided to go full yolo with it. Regrettably, I have not really implemented any SAGA or eventual consistency patterns. I’d really like to move to a message queue approach with something like RabbitMQ, which will hopefully give me an eventual consistency model. I decided to leave this refactor for sometime after release, since it would’ve been too much of a delay to change things over. Hopefully this will be one of the first back end improvements I do though (tech debt printer goes brrrrr)

The DevOps

Again, I’m F#cked :(

DevOps is equivalent to the boogie monster under your bed, and I’ve been hiding under the covers for as long as I could. I knew DevOps is something that would have to be done before I could even think of releasing. It is often overlooked as a requirement, but after working at a startup with no proper DevOps (for the first couple months at least), I can wholeheartedly say it is as important as Front end and Back end. Without proper DevOps, you have no idea what is running on production or staging. Having no insight into what’s running where or what’s going wrong when, is severely underestimated when it comes to business threats. I cannot stress this enough, YOU NEED proper DevOps! This is the part that makes your business go live!

Luckily I have my 10%er friend, who has recently become a full-blown expert in the field. The deal was, 10%er would get the staging environment up and running, and I will learn from it and get production up and running.

This included, but is not limited to, the following:

• Continuous integration: For continuous integration, we went with JetBrains Space. It’s a nice one-stop shop for everything you really need. Git, Maven packages, Docker images, automated builds, Task tracking etc etc.
• Continuous Deployment: This Is the difficult part, but with Kubernetes and ArgoCD it gets manageable.
• Monitoring: Everything gets pushed to Grafana Cloud, so that we can keep an eye on logs, metrics and traces.
• Infrastructure as code (IaC): Terraform is the way to go. <Insert screams after their business source license updates> I guess we’ll have to change to something like Pulumi in the future (tech debt printer goes brrrrrrrr)
• Hosting: 10%er had this brilliant idea of using Oracle Cloud Infrastructure (OCI). The really cool thing about this was that it provides a very generous free tier. The only caveat, AArch64 only, so he had to do some magic to build our images for AArch64 systems.

Overall, with the very generous free tier of OCI, 10%er managed to get our hosting fees down to something ridiculous like $5 per month. This is obviously not going to be sustainable once we need to scale more, but given the fact that we’re using terraform for our IaC, it’s as easy as a one line change to scale our nodes.

Getting all this to work was brutal. There are so many pain points that it’s impossible to list them all here. All I can say about DevOps is that I understand why they get paid so much more than normal software engineers. It’s because they have to take into account their extra day-to-day expenses they have. For example, they have to pay extra to visit a psychologist 3 to 5 times per week to keep them from going completely mental. Honestly, it’s the most frustrating thing ever.

But with all the struggles, come great rewards. It is a magical thing to see everything run swimmingly, autoscaling, auto deployments, no downtime, auto https, auto dns, iam policies, the list goes on and on. Everything just, works, and it’s beautiful.

Website

It’s also worth mentioning that I used Hugo static site generator to generate my website. I host the generated site with Netlify since it has a very generous free tier. A friend of mine has written a good blog post describing the same setup, feel free to give it a read

A Final Thought

This was a very high level view on our tech stack. If I were to list everything I had to consider in each category I’d be writing a book. The previous blog entry was WHY Adventure-IT is, and hopefully now you have an idea of HOW Adventure-IT is.

In the future, I might go more in depth into each category and hopefully give more insight for the tech-savvy out there, but not today.