
This is going to be a bit of an unusual post as it details a failed tinkering project.

I still want to write it however as I think its an interesting project that has promise and also has some relevance to my [recent](https://mikecann.blog/posts/chatter-craft-collaborative-art-with-just-your-voice) wo[rk wi](https://mikecann.blog/posts/tinkering-with-convex)th [Convex](https://mikecann.blog/posts/im-now-a-convex-developer-advocate).

# SpacetimeDB

What is [SpacetimeDB](https://spacetimedb.com/)? Well it touts itself as a "next generation serverless database."

It grabbed my attention earlier this year as it was focused on game development which is squarely in my wheelhouse.

The basic concept is illustrated in this rather rambling (and a little confusing) blog post: https://spacetimedb.com/blog/databases-and-data-oriented-design

[![](./spacetime-blog-post.png)](./spacetime-blog-post.png)

In summary they are aiming to create a realtime database that is a bit like ECS, but as (they argue) ECS can be thought of as a subset of a more traditional relational model, SpacetimeDB can be thought more of as a relational database.

So effectively think of it like Postgres but with some fancy sugar that makes it easier to work with.

In some respects this sounds a bit like convex except for that "realtime" part.

Convex is both an "application" layer and a "database" layer merged together combined with a bunch of developer-facing sugar that makes it great to work with.

The key difference between Spacetime and Convex is that spacetime is designed for stateful "realtime" applications (like games) whereas Convex is primarily a stateless serverless platform. It is possible to do things realtime-ish but as Convex [charges per function invocation](https://www.convex.dev/plans) it would get expensive really fast if you attempted this at scale.

# Pricing

This brings up the topic of SpacetimeDB pricing. Well its currently not mentioned. They have an open-source version of the platform that you can run locally (and presumably self-host) which is nice but if you dont want to mess with servers then they are going to offer a hosted version as is the database business model in vouge at the moment.

If I had to guess it probably wont be cheap to run, they might not even offer a free tier. Running multiplayer game servers has never been particularly cheap due to the high framerates combined with complex processing (such as physics) required.

# Tinkering

So onto the tinkering!

I had the idea I would build a realtime shared-cursor app where you can see everyone else's cursors move around perhaps click the screen and see pretty fireworks or particles or something. Very simple but should hopefully test some of Spacetime's features.

Before I get there however I felt it probably best to start with the quick-start tutorials and unfortunately this is where the troubles began.

You can develop the server-side component of your application in one of two languages, either Rust or C#. [Given my background](https://mikecann.blog/tags/c%20sharp) C# was the natural choice for me.

After copy-pasting the code (no ready-to-go example project unfortunately) from the [C# Quickstart](https://spacetimedb.com/docs/modules/c-sharp/quickstart) and running the `publish` command that pushes the built code to the locally running Spacetime platform I was greeted with the following error:

[![](./publish-error.png)](./publish-error.png)

Unfortunately I couldnt work out what this meant so I headed over to [their discord](https://discord.gg/SpacetimeDB) and posted about my issue. After a bit of back and forth of not fixing it it was suggested that I wait a few days for someone to look at it.

Sure enough a few days later they had [issued a patch](https://discord.com/channels/1037340874172014652/1134597033613869207/1183780061686546493) and the publish command now worked.

[![](./patched.png)](./patched.png)

Okay so now with the C# project in theory published I attempted to test it using their CLI:

```
spacetime call spacetime-sandbox send_message '["Hello, World!"]'
```

Unfortunately I now got a different error.

[![](./http-error.png)](./http-error.png)

I decided to move on, hopefully this was just a CLI issue I thought.

So I started a new [Vite project](https://vitejs.dev/) to get the client-side of the project working.

Spacetime provide client-side SDKs for a number of platforms: Typescript, Python, C# (Unity) and Rust. Although I suspect the most common usecase would be Unity or Rust I thought I would give Typescript a go as I have been using it most lately and I felt I would be able to debug any issues pretty quick.

So I started with the [Typescript Quickstart](https://spacetimedb.com/docs/sdks/typescript/quickstart) and unfortunately again they dont provide a ready-made example project so I went through and manually copied and pasted all the code as they described.

Unfortunately there were many issues in the provided code from non-idiomatic Typescript (use of `let` instead of `const` for example) and poor React structure (they suggest putting everything in a single component without any usage of Context or hooks) to simply not working example CLI commands (`--project_path` instead of `--project-path`).

The code required a lot of massaging but I think I had it all right, compiler issues fixed etc and attempted to run it.

I was now greeted with a new set of runtime errors in the console:

[![](./ts-errors.png)](./ts-errors.png)

Sigh. I suspect this might be something to do with the HTTP issue that I saw above.

I posted the problem to the discord once again but have yet to have a solution.

Just incase it was something to do with my local setup I decided to have a go and see if I could publish to their testnet which is their hosted version of the platform.

Unfortunately the [docs on the testnet page](https://spacetimedb.com/docs/deploying/testnet) were also not working as they say you should use the `spacetime server set "https://testnet.spacetimedb.com"` command but the CLI errors when you attempt this.

In fairness when I reported this to discord user "⚙ 3Blave" very quickly updated the docs so that it now reads `spacetime server add --default "https://testnet.spacetimedb.com" testnet`

So now you have to login to the spacetime testnet by calling `spacetime identity set-email <your-identity> <your-email>` but this command needs your `identity` which you get from `spacetime identity list` which again errors with:

[![](./identity.png)](./identity.png)

When I try to manually get my identity from the spacetime website (https://spacetimedb.com/login) I am greeted by a box that asks for my login email but nowhere to "signup" which is really strange. And when I attempt to put in my email I get another error.

[![](./signup.png)](./signup.png)

Quite frankly I was out of patience at this point and decided to end the tinkering here.

# SpacetimeDB Issues

I had planned to test out these things and make sure they were indeed issues before I wrote about them but as I was unable to get the code working correctly I only have reading the docs to go on. Apologies if I am off with any of these.

## Authorizations / Row Level Security

The first big issue I noticed with Spacetime is there is no concept of "Authorization" or "Row Level Security" that is there is no way that some information can be private from other users.

This is obviously critical for most (if not all) games. You need to hide some information from other players. For example in [BattleTabs](https://mikecann.blog/posts/battletabs-in-7-minutes) each player hides their ship positions from each other until the end of the battle, without this players could cheat by looking at the data that the server sends down or could directly issue queries to the database and retrieve that information.

When queried about this on discord I was informed that [that is indeed coming](https://discord.com/channels/1037340874172014652/1134597033613869207/1183210861700055150) and that there is a concept of "Private Tables" that you designate by prefixing your table with an underscore.

Without any docs on this im not sure exactly how this would work in a real game and how much work this might be for the user to implement.

# Migrations

I could find no mention of how DB migrations are supposed to work in their docs (despite their AI docs search assuring me there are 12 references)

[![](./search.png)](./search.png)

This to me is one of the MAJOR benefits of the way convex works you can very easily migrate your database by [changing the schema definition](https://docs.convex.dev/database/schemas) which then validates every single document in the the DB. This is a far more flexible way of structuring your tables than traditional relational databases but still ensuring the strong relational structure and type-safety enforced by your schema.

Im not exactly sure how SpacetimeDB works under the hood but they talk about replaying a Write Ahead Log, so im interested to know how you would go about evolving your schema over time

# Physics

I have big questions around how you would integrate this in a Unity project that requires server-authoritative physics or other game-engine features. Perhaps you are expected to implement those yourself? Perhaps you are supposed to run Unity headless server instances in parallel and query those from Spacetime?

Open questions for next time.

# Conclusion

This post is getting quite long and is getting quite negative. I dont want it to come across that way, I think SpacetimeDB is an interesting technology and is in a niche (realtime relational database targeted at game developers) that is very interesting to me.

I just feel like perhaps its a bit too early for the project and it needs to bake for a good while longer before its ready for further tinkering.

Keen to hear what you think if you have experimented with Spacetime, leave me a comment below :)



Just a short one this month as I have been working super hard with the team on the mobile port of [our game BattleTabs](https://mikecann.blog/posts/battletabs-in-7-minutes). But as this is a personal blog I thought I would share something I think is quite cool.

I have recently really been into the back-end platform [Convex.dev](https://www.convex.dev/) writing this review on it back in August: https://mikecann.blog/posts/tinkering-with-convex then entering the WebDevCody competition and coming third in this post: https://mikecann.blog/posts/chatter-craft-collaborative-art-with-just-your-voice.

So when Convex reached out to me a few weeks ago to let me know they were starting a brand new Developer Champion program and would like to invite me to be a part of it I was super excited and signed up immediately.

# What are the benefits?

From the Convex docs on the topic:

- Consistent and transparent AMA with leadership
- Exclusive Convex Champion Swag (coming soon)
- Individual [Stack](https://stack.convex.dev/) feature: Case study, profile or project feature
- Profile featured on a new champions page on [Convex.dev](http://Convex.dev)
- Sponsored trips to conferences if your presentation about Convex is accepted
- Complimentary Professional Plan for one team.
  - $0/month, in perpetuity
  - Up to five team members
  - Not for resale or transfer.
  - Includes only built-in monthly Professional resource limits.

Pretty cool!

# Whats involved?

Straight from Convex's docs on the topic:

- Creating blog posts or videos about Convex
- Attending meetups or conferences to give talks about Convex
- Promoting Convex, only if relevant
- Helping users be successful in the Discord

So what does it mean for me? Not much! Im still going to be focusing 99.9% of my time on Gangbusters but should I decide I want to start giving more talks its awesome that the program is there to help sponsor me if I were to give a Convex talk.

Ill still be writing blog posts on whatever interests me so theres no obligation there.

I really enjoy the power and speed of building things with Convex so no doubt you will be hearing a lot more about Convex powered projects on this blog, stay tuned!


I have got to be honest, I have never heard of [Web Dev Cody](https://www.youtube.com/@WebDevCody) before but when I saw that he had a [hackathon](https://hackathon.webdevcody.com/) in partnership with [Convex](https://www.convex.dev/) I knew I was going to have to carve off a bit of time from [BattleTabs](https://battletabs.com/) development to have a crack.



# Competition Categories & Ideation

[![](./categories.png)](./categories.png)

The competition was themed around Collaboration and AI so I racked my brain for a bit before coming up with an idea.

I wont spend too many words explaining what I built, instead ill let the video that I used to demo the project do the talking:

<iframe width="853" height="480" src="https://www.youtube.com/embed/XZ2iCEcW_io" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

# Results

Out of [53 submissions](https://hackathon.webdevcody.com/submissions), I came third in the AI category which im pretty happy with considering I spent less than a day on it.

Watch Cody talk about my project here:

<iframe width="853" height="480" src="https://www.youtube.com/embed/PJjk4Qg_GiU?si=9oUEhACACbj3KxkR&amp;start=186" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

# Conclusion

I had a ball working on this, its pretty astounding that you are able to build this much functionality in such as short time. I was able to implement and iterate as fast as I could go without any boiler-plate bogging me down.

[Convex](https://www.convex.dev/) really is an incredible technology, if you are looking to do a web app you should definitely check it out!

If you are interested in playing with Chatter Craft yourself the demo is here: https://chatter-craft.netlify.app/

If you want to peek under the covers a bit the source is available here: https://github.com/mikecann/chatter-craft


Its been a while since I have given a personal update so I thought I would take this month to talk about my recent move and some adventures with satellite internet.

# The Move

[![](./map.png)](./map.png)

So I have lived in Australia for 9 years now (cant actually believe that) and I have moved a total of 8 times. When I was living the UK I moved a similar number of times in my entire life. So its safe to say that im pretty comfortable with moving home.

One week ago however we moved house from Perth to Busselton (Western Australia) and this time was different. This time for the first time in my life we had furniture and a 3 year old to contend with.

We knew we weren't likely to fit everything in our revivalists van so we decided to hire a transit van for a day 2 days before the big move day. Fortunately my parents were on hand from the UK to help with the tetris-like packing.

[![](./tetris.jpg)](./tetris.jpg)

So then the big day came and most of the items fit in the van but unfortunately not everything.

[![](./van.jpg)](./van.jpg)

[![](./left-over.jpg)](./left-over.jpg)

So this meant that the following day I had to hire a trailer and drive back to Perth to collect the remaining items.

[![](./trailer.jpg)](./trailer.jpg)

With everything now moved we could finally start to settle in and enjoy the environment.

[![](./garden.jpg)](./garden.jpg)

<iframe width="853" height="480" src="https://www.youtube.com/embed/T4kaZX1ZzzM" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

# Starlink

So as I work from home on [BattleTabs](https://mikecann.blog/posts/battletabs-in-7-minutes) its essential that I have internet and preferably good internet.

Although Busselton is 3 hours south of the capital of WA Perth, it isnt a total backwater. It does have broadband connection in the form of the National Broadband Network (NBN).

The only issue is that when I called up to move my connection the customer support staff informed me that it was likely I was going to get poorer speeds that I was currently getting. I was paying $79 per month for a 50Mbs down 20Mbs up connection (50/20) this is unfortunately the limit of copper cable broadband and although Fibre NBN is rolling out across WA, neither my Perth home had it nor the new place in Busselton.

[![](./aussiebb.png)](./aussiebb.png)

Worse than that I was informed that I was likely not going to even get the 50/20, likely the maximum would be 50/13.

This wasnt great for me who lives on the internet most of the day, so I decided to look into alternatives. The only real contender was [Starlink](https://www.starlink.com/), Elon Musk's internet from space company.

I had reservations about going Starlink as [I had read](https://www.whistleout.com.au/Broadband/Guides/Starlink-Australia-Everything-you-need-to-know) that the upload speeds wouldnt be much faster than I was due to get.

It was a combination of speaking to a friend who assured me that was likely old information (Starlink speeds improve as more satellites are launched) and seeing the 30-day-money-back-trial paired with the 50% discount on the dish for rural customers (me), I decided to take the plunge and order one.

[![](./30day.png)](./30day.png)

So after we moved down here I got the NBN internet setup and it was worse that I had feared. The following was just about the maximum speed I could achieve over a couple of days of testing:

[![](./nbn-speed.png)](./nbn-speed.png)

Less than 5Mbs upload! This was definitely not going to cut it.

Time to crack out the Starlink.

[![](./starlink-box.jpg)](./starlink-box.jpg)

After a bit of faffing around with trying to find a temporary location for the dish that wasnt in the way but also still allowed us to lock the house at night I was ready to run some tests.

[![](./dish.jpg)](./dish.jpg)

So even though the dish is only in a temporary spot and is partially blocked by some stuff down the side of my house the speeds are far faster than I was expecting!

[![](./starlink-speed1.png)](./starlink-speed1.png)

[![](./starlink-speed2.png)](./starlink-speed2.png)

The speeds are definitely more variable than NBN, im assuming this is just the way it works as it swaps between satellites. Im hoping it will also stabilize a bit over time and once I have it mounted on the roof.

But my calculations it is on average 6-10 times faster than the fastest NBN I could get here and incredibly it even has lower latency!

So all in all I have been mightily impressed with the service so far. For $139 per month (plus the $300 for the dish) its more than I was paying before but not much more than the comparable NBN speed (250/25) thats assuming I would have ever been able to get those speeds.

I have received quotes from two TV / Satellite dish installers that it will cost me between $300-$600 to get it properly installed. I however noticed that we have an old satellite dish on the roof already so I might have a crack at just repurposing that mount myself and thus hopefully save some money.

# Conclusion

Im really happy to finally have moved into my new home and super excited that I have internet blessed by his holyness, Elon Musk :)



As you may be aware we have been working really hard lately on the next version of our game [BattleTabs](https://mikecann.blog/tags/battletabs).

In an effort to broaden the game's reach for the upcoming mobile release, we decided to invest some time in producing "devlogs".

If you're not familiar with devlogs, they are a popular format on YouTube where developers discuss their past and current progress with a game.

We hope that viewers will find the video entertaining and informative about the game. Ideally, they will subscribe and follow along as the game progresses and future devlogs are released.

With that in mind, we created our first devlog. This one is longer than future ones because I was eager to tell the story up to this point. Watch it and let us know what you think!

<iframe width="853" height="480" src="https://www.youtube.com/embed/kDlEoLjflGw" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

We were initially hesitant about investing the time (about 5 days) to prepare and create this video, as that's time I could have spent working on the game.

However, we justified the time investment because marketing is an area where none of us on the team are experts. We feel the need to explore every possible avenue to spread the word about the game and attract new users.

Speaking of which, we are taking [BattleTabs to the Perth Pixel Expo](https://www.pixelexpo.org.au/game-devs) this year, where we have a double-wide booth.

[![](./expo.png)](./expo.png)

I'm looking forward to showcasing the game in person to a larger audience, running tournaments, and giving out stickers and t-shirts!

We plan to continue our devlog videos roughly monthly, depending on other commitments and the content we have to show off.

Anyway, let us know what you think in the comments. If there are things we can improve for future videos and, importantly, other communities you think might be interested in this video, we'd love to hear from you!



Strap yourselves in, this one is a bit of a rant.

For the longest time I have been a Fitbit Versa fanboy. I bought the Versa 1 back in June 2018 and have replaced and upgraded ever since.

When my Versa 2 last stopped working a couple of months ago (due to water ingress), Fitbit once again offered to either replace my watch with a Versa 2 or get a shop voucher for the same amount. I this time decided I wanted to upgrade to the recently released Versa 4.

If you weren't aware Google purchased Fitbit back in Jan of 2021 and the Versa 4 is the first watch they released in the Versa line since that acquisition.

I thought that Google would improve the line and not instead kneecap it instead in favour their competing [Pixel Watch](https://store.google.com/au/product/google_pixel_watch) line...

Surely Google has purely [altruistic reasons](https://time.com/5717726/google-fitbit/) for purchasing Fitbit I thought..

Well I received my Versa 4 and quickly realized that past-Mike is a poor naive fool.

## No Custom Watchfaces or Apps

I have long enjoyed creating and running my own watchfaces on my Versa. I even wrote about how much I enjoyed the process [back in May 2021](https://mikecann.blog/posts/minimal-mike-a-fitbit-watchface).

Unfortunately however Fitbit / Google in their infinite wisdom have decided to [no longer allow this](https://community.fitbit.com/t5/Versa-4/Versa-4-can-t-install-any-apps/m-p/5293897/highlight/true#M2012).

By the way, this isnt mentioned anywhere. You have to go spelunking through their forums to find this out. Their official developer docs [are suspiciously quiet](https://dev.fitbit.com/getting-started/) on the topic only mentioning the Versa 3. There are [A LOT of annoyed people](https://community.fitbit.com/t5/Product-Feedback/3rd-Party-Apps-on-Sense-2-and-Versa-4/idi-p/5248677) if you go looking.

BTW this also applies to Apps. You can no longer create custom apps so[ I am not able to pair](https://docs.woosports.com/docs/supported-devices) my [Woo](https://woosports.com/) kitesurfing jump-tracker with my Versa 4 like you can on [Garmin devices](https://apps.garmin.com/en-US/apps/53d9c500-614e-4e3a-88b6-9f65bc6a144a)..

[![](./fore.jpg)](./fore.jpg)

# Automatic Exercise Mode Is Terrible

In principle, having the watch automatically detect that im doing exercise and only then enable battery-hungry features such as GPS and blood-oxygen monitoring is a great idea.

The only problem is when it is implemented as it has been on the Versa 4 then its worse then useless, it is a massive problem for the device.

<iframe width="419" height="744" src="https://www.youtube.com/embed/y_D1cA-IWw4" title="How Bad is the Versa 4?!?" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

Imagine you start to do some exercise, go on a walk or play squash or anything. The watch detects that you are exercising, great! Now you also want to see what the time is.. uh oh..

That right, you can no longer use your watch to tell the time once it detects that you are exercising. The only way to return to "watch" to see the time is to exit the exercise mode and this then brings up the next major issue..

As you can see from my video above, the screen doesn't work if you fingers are even a little damp. What happens when you start to exercise? Thats right, you start to sweat, rendering the tiny touchscreen inoperable.

You eventually you manage to press the tiny "end exercise" button and continue on with your exercise. 5 minutes (or so) later the watch automatically detects exercise again ARRRGGGGHHH!

P.S. As far as I was able to ascertain there is no way to disable this "functionality".

## Always on Display is Pointless

One of the main reasons I have always had for purchasing a Versa over an Apple or Samsung watch is the multi-day battery life. I used to enjoy that back when I used to wear the [e-ink Pebble watches](<https://en.wikipedia.org/wiki/Pebble_(watch)>) and didnt want to give it up.

One thing I did miss however was the always-on display.

Having to "flick" my wrist to see the time was an "okay" solution tho it didnt always work. It also used to activate randomly in bed which would cause much marital frustration as a bright light would shine in her face during the night.

So when I saw that the Versa 4 supported an "always on" display I was super excited as I felt that hopefully they had solved these issues.

Unfortunately however I quickly discovered that turning on this feature rapidly drained the battery so that the average charge time went from 5-7 days down to 1-2 days which made me question what is the point?

I know technologically it's possible as every Garmin watch has an always-on display and maintains weeks or months of battery life.

# Annoying Notifications

[![](./notifications.png)](./notifications.png)

I couldn't find any good footage of this issue and for some reason my Versa has decided to stop sending me notifications at all (again) so just take my word for it, getting notifications on this device are super annoying.

Im sure some designer at Fitbit thought it would be cute if they played an intro animation every time you get a notification before actually showing you the contents of that notification. The issue is this intro takes like 2 seconds.

This is annoying particularly when you want to quickly scan that text message while continuing what you are doing.

Like with most other things with this watch, this is not configurable without entirely disabling notifications.

# Update Resets Settings

This one isn't a Versa 4 specific issue and has always happened to me on Fitbit devices but I thought I would mention it while im on the rant-train.

As mentioned in my point above for some reason my watch has decided to stop sending me notifications. Why? Who knows. Perhaps it updated? That seems to always reset my "reminders to move" settings (super annoying btw). Probably the only way to remedy this is to unpair the watch and repair it.

If that doesnt work, factory-reset the watch and setup everything again (yes I did read that as an official way to fix it but cant find the link).

# Conclusion

So I could have titled this post "Dont Buy a Versa 4" but unfortunately I cant see this situation getting any better with future Fitbit products. Google is likely going to continue to favour its own brand of smartwatches over Fitbit. Having now acquired the competition it will strangle then eventually kill it.

So after all this I decided to change over to to Garmin smartwatches. If you have gotten this far too and are now scared off Fitbit devices I can definitely recommend Garmin. They have a massive range of options, all the apps, simple to use, custom watchfaces, physical buttons, no touchscreen, GPS, always on display week+ battery times, just about everything is better.



AI is all the rage at the moment and for good reason, its going to revolutionize many industries.

Since I first wrote about it in the context of Game Development back [in November](https://mikecann.blog/posts/ai-and-the-future-of-game-development) I have been thinking how we can apply some of the advances to [our](https://gangbusters.io/) game [BattleTabs](https://battletabs.io/).

# Problem

Since the first version of the game (3+ years ago) we allowed users to name their player. This, we feel, gives a better sense of ownership and attachment to the game.

[![](./signup.png)](./signup.png)

A year or so after launch [we introduced the "Custom Fleets" feature](https://mikecann.blog/posts/battletabs-ships-and-fleets-update). As part of this feature we also allowed the user to name their fleet.

[![](./fleet-name.png)](./fleet-name.png)

Again we feel this creates a stronger feel of attachment to the game and their choices within it.

Both of these features however opened up an annoying issue for us: profanity.

We started to get reports of players being offended by the names of other players.

[![](./early.png)](./early.png)

This was an issue as lots of children played BattleTabs so we felt like we had to do something about it.

# Local Filtering

So.. we started to implement a "naughty words" list.. which grew.. and grew and grew..

[![](./list.png)](./list.png)

No matter how many words or phrases we put on this list, users always seemed to find ways to be offensive.

Did you know for example that "Tangina nyo" is a swear word in Fillipino? Nope, neither did I.

# Community Filtering

So after many months of this we decided to enlist some help.

We implemented a "Report Player" button on the player profile page.

[![](./report.png)](./report.png)

This then sent a "Report" to a special channel in discord that only us and our Community Moderators could see.

[![](./reports.png)](./reports.png)

We then added special discord commands to our bot that allowed our moderators to rename the player and then potentially ban then from renaming themselves again (repeat offenders).

This worked.. kind of.

Obviously it relied on people reporting others and then out Community moderators being available to handle the report.

So it meant that some names slipped through the cracks or went days before being detected.

# AI Enters The Chat

Enter ChatGPT in late 2022 and suddenly every problems seems possible to solve with the hammer that is LLMs.

So we decided to do some quick experimentation and see if AI could detect profanity in a phase.

We created a sample spreadsheet of a sample of profanities and non-profanities and hooked up ChatGPT plugin to Google Sheets and tried out a few prompts.

[![](./sheet.png)](./sheet.png)

It definitely _seemed_ to work. Enough that it was worth going ahead and implementing it in the code.

Ill save you all the iterations we did around the prompts and just show you what we ended up with:

```ts
{
  max_tokens: 20,
  temperature: 0,
  messages: [
    {
      role: "system",
      content: `I want you to act as a profanity checker. I will give you potentially profane word or phrase and you will tell me if it is profane or not. Do not provide personal opinions or assumptions. Do not respond asking how you can assist. Simply respond with the words 'profane' or 'not profane'. If it is profane you may also give a very short explanation of why you think it is profane.`,
    },
    {
      role: "user",
      content: `the phrase is: '${phrase}'`,
    },
  ],
  model: "gpt-3.5-turbo-0301",
}
```

We set the temperature to very low as we wanted it to give consistent outputs given the same inputs, but im not sure is that makes much difference.

We have quite a long prompt, this was based on recommendations from [awesome chat gpt prompts](https://github.com/f/awesome-chatgpt-prompts).

We also added the `the phrase is: '${phrase}'`, instead of simply providing the phrase because for some reason the AI would still get confused and try to respond to the phrase rather than the system message:

[![](./whoops.png)](./whoops.png)

For example in the above you can see that if the player provides their name as "yes" then the AI responds with "Great, let's begin. What is the first word or phrase you would like me to check?" instead of the expected "not profane".

Its not perfect and we are still refining it but it generally works really well and has caught thousands and thousands of profane player names and fleet names.

[![](./results.png)](./results.png)

# Conclusion

In the end im super happy with how this has turned out. For minimal effort and cost we were able to get a much more super profanity checker improving the experience for the 99.9% of our players who dont want to see profanity while playing [BattleTabs](https://mikecann.blog).



(no this isn't an april fools post :P)

I wrote my first Application 31 years ago when I was [7 years old](https://mikecann.blog/about). Since then I have spent most of my career learning the fine art of of crafting clean code. From hand-crafted HTML and CSS to wrestling with MXML and AS3 I have been doing this for a long time.

With the incredible advances in Large Language Models (LLMs) and their ability to generate code on demand i'm starting to wonder if the days of hand-crafted applications are numbered.

# What is a LLM?

You must have been living under a rock if you haven't heard of ChatGPT over the last few months, but just in case you are a granite hermit:

> A LLM is a large neural network that has been trained on a large corpus of text. The most famous of these is [GPT-3](https://openai.com/blog/openai-api/) which has been trained on 175 billion words of text. The idea is that you can feed it a prompt and it will generate text that is similar to the prompt. For example, if you feed it the prompt "Describe the life and work of Albert Einstein", it will generate a text that provides information and details about Einstein's life and contributions to science.

(yes the above was generated using a LLM)

# What is an Application

For the purposes of this post, let's say that an application is made up of three parts:

1. The persisted state or "data"
2. The interface into that data (API). This is generally your "server".
3. The User Interface that makes use of the API. This is generally your "client"

I think that each of these parts is going to undergo a massive transformation thanks to the recent AI advances.

As its most familiar to most people lets start with the UI.

# What is a UI?

A UI is a User Interface. It is the thing that you interact with to use an app or program. Almost every app has one, think excel or discord:

[![](./excel.png)](./excel.png)

The only real examples I can think of that don't are things like CLIs (Command Line Interfaces), but even then its arguable if a CLI even if its just text based.

One way to think about them is they are the visual front end on whatever business logic and data that makes up the core of you application. It allows you to query and mutate that data in a way that is easy for the user to understand.

# UIs of Today

Current UIs are generally hand-crafted by humans to look and feel a certain way. These usually are a very labour intensive pursuit on the behalf of the developer.

If you don't like or understand the particular design of a given UI, you are generally out of luck because that's the way it's been developed.

# UIs of Tomorrow

Back at the start of [November 2022](https://mikecann.blog/posts/ai-and-the-future-of-game-development) I wrote a post on how I thought AI would change the future of game development. In that post I talked about how I thought that the future of app development will feel more like sculpting than coding.

> I have spent so much of my career thinking about and worrying over the best practices for programming. Am I correctly implementing the SOLID principles in OO C# or and I fully understanding the Monad or ADTs in Functional Programming? In the future IT DOES NOT MATTER. The output is the only thing that matters.

> I think about the future of programming more as sculpting. You start off with a blank slate then using your own words you command by command iterate towards your end goal.

This was only five short months ago, but so much has happened since then. ChatGPT was released and became [the fastest growing user base of all time](https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/).

The world suddenly realized that you can get a long way with just a text-based conversation via a chat window

This revelation caused [Microsoft to double down on their investment into ChatGPTs creator OpenAI](https://www.afr.com/technology/microsoft-invests-14b-in-chatgpt-maker-openai-20230124-p5cex0) and to very rapidly [begin implementing chat-like interfaces](https://blogs.microsoft.com/blog/2023/03/16/introducing-microsoft-365-copilot-your-copilot-for-work/) into all their products.

You can now simply ask an assistant for information using natural language, and it can go out to the web, find that information, summarize it, and contextualize it within a few seconds. This has many questioning why would you ever need to use a search engine?

Well, this combined with the double-pronged threat of vastly reduced cost of producing generated fake but believable content, caused Google to [declare a "Code Red"](https://www.forbes.com/sites/davidphelan/2023/01/23/how-chatgpt-suddenly-became-googles-code-red-prompting-return-of-page-and-brin/?sh=720f5a2b5977) to [frantically rush](https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/) to catch up.

These new new LLMs aren't just good at generating text however, they are also really good generating code too.

So what has all this got to do with User Interfaces? Well if you put it all together im imagine a world where an application is simply its data and an API. The User Interface is generated on demand by a LLM.

For example take a look at Microsoft's recent [release of Security Copilot](https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/):

<iframe width="560" height="315" src="https://www.youtube.com/embed/g1HoXNoP3V0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

The UI for Security Copilot is simply a chat prompt. The has a conversation using natural language to get the information they need.

[![](./security-copilot.png)](./security-copilot.png)

The requested information is returned and decorated using Microsoft's own user interface controls and styling.

I believe this is just the start of what is going to happen.

Imagine instead of simply returning the information requested, the AI that lives inside your application will also return some UI code that can be used to render that information.

It's important to note that this code might be made up of building blocks that the developer has specified so that the application retains the same look and feel. This however isnt entirely neccessary as the LLM will know how to generate a good UI to represent the information that it contains.

In this future applications become much more personal and are tailored to the way you want them to work.

Sure there are going to be occasions where you just want to get the experience that the developer intended, I can imagine there might be some pre-defined layouts but then you are free to customize it to your hearts desire.

We [at Gangbusters](https://www.gangbusters.io/) use the excellent tool [Metabase](https://www.metabase.com/) to query and visualize our data.

[![](./metabase1.png)](./metabase1.png)

Its a fantastic tool with a great interface for "asking questions" of your data and then visualizing the results.

[![](./mb-questions.png)](./mb-questions.png)

But it could be better, what if you could just use natural language to ask those questions and instead of simply showing a predefined pie chart or graph, instead it could tailor the UI to whatever best suits the data. It could provide buttons and controls to allow you to manipulate the data.

This general thing is how I imagine application UIs will work in the future.

If we take it to its logical conclusion then perhaps the operating system's UI itself is generated dynamically on demand. There are no "applications", the AI that runs in the OS simply goes out to the various services on behalf of the user, returns with the data and generates a UI to display it on the fly.

# The Future of APIs

This is a bit of a tangent but:

UIs need to render some data to the screen. They get that data from a data source that they interface with using an Application Programming Interface (API).

A normal website is a UI that is communicating with one or many servers via an API generally REST or GraphQL.

These APIs are generally written by humans so that they are easy to understand by other humans.

In the age of generated UIs I think this will change. APIs no longer need to be written for human consumption primarily they can instead by written for machine consumption. This will mean a lot less work for the developer that has to create and document an API for other humans to consume.

Perhaps the API will simply be a text prompt. You will in natural language simply ask for the data and then an AI that lives in your application will understand that, convert it into an SQL statement and then return the data to the UI.

[![](./sql.png)](./sql.png)

We can add "system" prompts to our AI to ensure that it doesnt return any private data or allow it to manipulate data in any undesirable ways.

# The Future of Data Storage

So you have an AI generating your UI and the API is simply a prompt, whats left? Well just the data storage part.

In the section above I imagine a future where an AI is able to generate an SQL query to interrogate an existing relational datasource such as Postgres. But what if we took that one step further.

What if we wanted the data storage to be more flexible so that as the application evolves the new structure of the data can be stored correctly?

Currently we have to define migrations to modify the schema of our database as the application evolves over time but what if instead data was stored in a less-structured NoSQL like manner. Now your data storage can change over time without and migrations.

The AI can still write NoSQL queries to access the data and it could also record a "dynamic" schema that it uses before it queries the data source.

Now if we take that another step forward, what if the AI is able to run in the background and spot inefficiencies in the way the data is structured and restructure it on the fly?

In this world you no longer know how your data is stored as the AI has organized and evolved it as your application has evolved.

But we can take this a step further. What if your data isn't structured at all? This is a future suggested [by @DYtweetshere in January](https://twitter.com/DYtweetshere/status/1617471632909676544?s=20).

[![](./backend.jpg)](./backend.jpg)

Instead the the AI "backend" simply "remembers" all the previous prompts ever given and is able to infer structure and meaning from that data as new prompts come in.

This is the ultimate in storage flexibility as there is now no longer and structure to your data, it is instead simply a list of strings.

Now this final example will take a few more advances in model context size as the AI must keep everything ever muttered in its "memory" so its can draw meaning from it without querying it. For a large app this is currently infeasible, but I suspect possible in the future.

The other potential is it stores a "compression" of the data via weights in exactly the same way the AI itself is trained but that by its very nature would be a lossy mechanism and thus your stored "Todos" (for example) would only have a probability of being recalled exactly which might not be ideal.

# Conclusion

For the past 31+ years I have been making applications in generally the same way. Hand-crafted some code for the UI, API and Data store, refine and repeat. I think this age is finally coming to an end.

I think the next few years are going to see some of the biggest changes in the way applications are created and consumed. More so than the cloud computing revolution, more so than even the mobile computing revolution.

I for one am excited about the potentials for these changes and look forward to what the future brings!



Are you a TypeScript enthusiast looking for ways to streamline your development process? Look no further! As someone who has been writing TypeScript [for a significant amount of time](https://mikecann.blog/posts/post-to-tumblr-5), I have accumulated a handful of utility tools that I consider essential for every new project. These tools have proven to be so helpful that they have become a staple in my development toolkit. In this blog post, I will share with you some of these tools that will make your TypeScript development journey smoother and more efficient.

# ensure

If you're familiar with JavaScript, you know that null and undefined can be a source of frustration and errors in your code. However, one of the biggest strengths of TypeScript is its null safety. TypeScript's type system provides a way to catch null or undefined values at compile-time, before they can cause runtime errors. This not only saves time and effort in debugging, but it also helps to ensure the reliability and stability of your code.

There are however occasions when the program cannot continue if the value is null or undefined. Thats where `ensure` comes in:

```ts
export const ensure = <T>(
  obj: T | null | undefined,
  err = `variable was null or undefined when it shouldnt have been.`
): T => {
  if (obj === null) throw new Error(err);
  if (obj === undefined) throw new Error(err);
  return obj;
};
```

I use this super frequently when I know that something must exists within an array otherwise the application cannot continue;

```ts
const someListOfThings: string[] = ["bar", "foo", "moo"];
const maybeFound = someListOfThings.find((s) => s == "foo");
//       ^ is now string | undefined but we know it must be there
const found = ensure(maybeFound, `couldnt find 'foo' in the list`);
```

Its basically an assert but happens on one line with a optional nice error message;

# sequence

I frequently want an array of something of a given length. This helper function is super useful for creating a new array of the given length and then populating it with a sequence of those numbers.

```ts
export const sequence = (count: number): number[] => {
  return [...Array(Math.floor(count)).keys()];
};
```

This is useful in all manner of ways from testing to UI. Here is an example where I use it in [BattleTabs](https://battletabs.com) to render a given number of clouds (as part of the background):

```tsx
<div>
  {sequence(numClouds).map((i) => (
    <Cloud key={i} />
  ))}
</div>
```

# mapObj

Everyone knows about array.map but I often find myself needing to map over an object too. That is I want to keep the keys the same but change the values. Thats where the next util comes in:

```ts
export const mapObj = <T extends string, U, V>(
  obj: Record<T, U>,
  mapper: (key: T, value: U) => V
): Record<T, V> =>
  Object.fromEntries(
    Object.entries(obj).map(([key, value]) => [key, mapper(key as T, value as U)] as const)
  ) as Record<T, V>;
```

An example of using it might look like:

```ts
interface Person {
  name: string;
  age: number;
}

const people = {
  person1: { name: "Alice", age: 30 },
  person2: { name: "Bob", age: 25 },
};

const peopleAges = mapObj(people, (key, value) => value.age);
```

# filterObj

Similar to mapObj its often useful to filter some entries from an object:

```ts
export const filterObj = <T extends string, U>(
  obj: Record<T, U>,
  predicate: (key: T, value: U) => boolean
): Partial<Record<T, U>> =>
  Object.fromEntries(
    Object.entries(obj).filter(([key, value]) => predicate(key as T, value as U))
  ) as Partial<Record<T, U>>;
```

And an example of its use would be:

```ts
const filtered = filterObj(people, (key, value) => key == "person2");
// { person2: { name: 'Bob', age: 25 } }
```

# wait

Its often useful to wait a given amount of time for something:

```ts
export const wait = async (ms: number): Promise<void> => {
  return new Promise((resolve) => setTimeout(resolve, ms));
};
```

an example of its use might be:

```ts
export const turnTakingSequence = (player: Player, ai: AI) => {
  await player.takeTurn();
  await wait(1000); // Add a nice delay
  await ai.takeTurn();
};
```

# match and matchKind

This is a bit of a cheeky one as its not really my util function but I have a full blog post on this "pattern matching" technique here: https://mikecann.blog/posts/discriminated-unions-and-pattern-matching-in-typescript

Go there to see what the function looks like an more depth but its a suuuper useful util that I use everywhere in my projects. An example of its use might be:

```ts
export type GeneralJob =
  | MatchmakeJob
  | UpdateUserDerivableMedalsJob
  | RefreshUserSubscriptionJob
  | CollectUncollectedBattleRewardsForUserJob
  | UpdateUserScoresFromMatchFinishJob
  | AITakePlayerTurnJob;

export const createGeneralJobsHandler = ({ services }: { services: Services }) => (
  job: GeneralJob
) =>
  matchKind(job, {
    "update-user-derivable-medals": ({ userId }) => updateUserDerivableMedals({ userId, services }),
    "refresh-user-subscription": ({ userId }) => refreshUserSubscription({ userId, services }),
    "collect-uncollected-battle-rewards-for-user": ({ userId }) =>
      collectAnyUncollectedBattleRewards({ userId, services }),
    matchmake: () => services.backgroundProposalMatchmaking.check(),
    "update-user-scores-from-match-finish": ({ matchId }) =>
      updateUserScoresFromMatchFinish({ services, matchId }),
    "ai-take-player-turn": ({ userId, matchId }) =>
      services.aiTurnTaking.takeTurn({ userId, matchId }),
  });
```

# randomIndex, randomOne and randomFew

```ts
// Replace with whatever random number generator you want (seeded or otherwise)
const getRandom = () => ({ nextNumber: () => Math.random() });

export const randomIndex = <T>(items: T[]): number =>
  Math.floor(getRandom().nextNumber() * items.length);

export const randomOne = <T>(items: T[]): T => items[randomIndex(items)]!;

export function randomFew<T>(items: T[], count: number): T[] {
  let input = [...items];
  const output: T[] = [];

  for (let i = 0; i < count; i++) {
    if (input.length == 0) return output;
    const index = randomIndex(input);
    output.push(input[index]!);
    input = removeAt(input, index);
  }

  return output;
}
```

These ones are super useful for returning one random element from a list or multiple. For bonus points you can configure the random number generator yourself so it becomes deterministic (useful for testing).

# randomRange and randomIntRange

While we are talking about random things, these two are super useful:

```ts
/**
 * Returns a random number between min (inclusive) and max (exclusive)
 */
export function randomRange(min: number, max: number): number {
  return getRandom().nextNumber() * (max - min) + min;
}

/**
 * Returns a random integer between min (inclusive) and max (inclusive).
 */
export function randomIntRange(min: number, max: number): number {
  min = Math.ceil(min);
  max = Math.floor(max);
  return Math.floor(getRandom().nextNumber() * (max - min + 1)) + min;
}
```

# wrap

And one last one

```ts
export function wrap(indx: number, max: number) {
  return ((indx % max) + max) % max;
}
```

This is useful for wrapping a given value within a known range, e.g.:

```ts
export function rotateDimension2DCW(
  { width, height }: Dimensions2D,
  degrees: number
): Dimensions2D {
  const wrapped = wrap(degrees, 360);
  if (wrapped % 90 != 0) throw new Error(`cannot rotate a dimension by a non 90 degree angle`);
  if (wrapped == 0 || wrapped == 180) return { width, height };
  return {
    width: height,
    height: width,
  };
}
```



I have been a long fan of Heroku and we have been using it as the platform that powers [BattleTabs](https://battletabs.io/) for over 3 years and before that [Markd](https://markd.co) but they have long since stopped innovating and after a number of stressful events I decided it was time to break up.

# TLDR;

Ditch Heroku, move to fly.io, thank me later.

# Pain

Moving a Database to another platform is never an easy thing to do so I resisted it for the longest time despite knowing that we were paying way over the market price for the service. It just wasn't worth the developer time and potential risk of moving an unknown platform. Sometimes its "better the devil you know".

But then the issues started happening.

Every now and then, for no apparent reason, the database would grind to a crawl. Queries would start taking minutes to execute before eventually timing out. This then put backwards pressure on the web-servers as they buffered requests and started to suffer memory issues.

And of course, this all would happen just before I was going to bed at night.

My thinking at the time was perhaps we were outgrowing the database.

You see, Heroku doesnt really offer much in the way of metrics for your database. Sure they have the Postgres dashboard but that thing is pretty minimal at best and doesn't tell you anything about the CPU or memory pressures that the DB is under.

My solution each time this happened was to simply make the DB larger.

We started working our way through [the plans](https://elements.heroku.com/addons/heroku-postgresql#pricing) progressively getting larger and larger until we ended up paying $400 per month just for our Postgres instance.

I was getting more and more frustrated and nervous about this because I had no idea what the fundamental issue was.

Eventually I opened a support ticket with Heroku and they told me to try to use the `heroku pg:diagnose` tool, so I did and I got the following:

[![](./diagnose.png)](./diagnose.png)

Notice that one item in yellow there "Disk Burst".

What is disk burst I ask myself? I didnt see anything on Heroku's [pricing page](https://elements.heroku.com/addons/heroku-postgresql#pricing) about IO limits

[![](./pricing.png)](./pricing.png)

So I ask the Heroku Support and well..

[![](./support.png)](./support.png)

Okay so Heroku has "hidden" IO limits that I was totally unaware of before and we are hitting this on certain tables and it is silently throttling us with no errors or warning or anything.

So in the end I ended up bumping up the size of the DB again but by this point I had had enough, it was time to move on.

# Moving

I started looking around at options and oh boy there are a lot of options for Postgres.

I'll save you hours and hours of research and comparison and just say that I ended up landing on [Fly.io](http://fly.io/).

The [pricing](https://fly.io/docs/postgres/) is just insane we can quite literally get 6 databases of the same size as Heroku's or one massive 8-core 32GB machine for the same price.

The price is definitely nice but thats not the main reason we moved, the main one was to get away from the hidden limits and poor metrics. Thankfully Fly.io really excels in this regard providing not only a basic metrics dashboard but a MUCH more complete Grafana dashboard.

[![](./basic.png)](./basic.png)

[![](./grafana.png)](./grafana.png)

Through that I can get a really clear view on how much memory, CPU, and importantly, IO the DB is using.

[![](./io.png)](./io.png)

You can even setup alerts to let you know if things go out of whack.

What this information let me do was to see that we actually didn't need all the resources we were paying for on Heroku and so once the arbitrary IO limit was gone I reduced the size of the DB down from 8GB down to 4GB and the disk size down from 256GB to 40GB.

TBH I think we could even go smaller if we wanted to play it a bit more riskily.

# Cons

So its not all sunshine and unicorn farts of course.

Fly.io is NOT a managed service [as their website](https://fly.io/docs/postgres/getting-started/what-you-should-know/) takes great pains to highlight. This means upgrades and downtime are all down to you.

Its also quite a bit more work to get things setup just right. For example you have to set up your own external IP if you want to access your database from outside of your applications.

Fly is still pretty young and its platform is constantly moving. This can be both a good and bad thing. There are some rough edges in places but new features and upgrades are constantly being added and the devs are very responsive to customer feedback.

# Conclusion

So in the end if you start off paying $400 per month and reduce that down to $20 per month you can get a 20x saving on your DB costs AND get a peaceful rest at night knowing that you have lots more visibility into whats going on and alerts to let you know when things go pair shaped.

Hopefully this is the last database migration we ever have to make for the game!



I dont know about you but every time I have to write a CI script or modify an existing one my stomach fills with dread. I just know im in for a long frustrating session of fighting with some tortured mutant DSL written in YAML only to wait 15 minutes between iterations of cryptic error messages.

Why oh why cant things be better?

# YAML

## Problems

In in the web development world some of the most popular CI platforms today are Github Actions, Jenkins, CircleCI and TravisCI. The one thing they all have in common is they use the [flaming ball of excrement that is YAML](https://noyaml.com/).

Why on earth to these CI platform all choose YAML? I have no idea.

The thing is, if it was just left at YAML that would be bad enough ([for](https://noyaml.com/) l[ot](https://news.ycombinator.com/item?id=26234260#:~:text=YAML%20is%20one%20of%20the,of%20various%20line%2Dendingness'.)s [of](https://sehun.me/why-you-hate-yaml-and-how-to-tackle-it-3e8471cf1ca8) reasons) but what invariably ends up happening is the platform developers add features things such as variables, functions and string interpolation. These are concepts that really dont belong in a declarative language and end up as hacky messy balls of confusing text.

Dont belive me? [Go take a look](https://circleci.com/docs/reusing-config/#authoring-reusable-commands) at the tortured syntactical hoops you have to jump through to define a "reusable" function that can take parameters in CircleCI.

Don't even get me started about the nightmare that is string interpolation in CircleCI or Github Actions YAML. Inserting environment variables into build commands and other scripts should be the bread and butter of any CI system but its unnecessarily complex and convoluted and error prone in a declarative format such as YAML.

I posted the question to Stack Overflow ["Why do most CI / CD platforms use YAML for their pipelines?"](https://stackoverflow.com/questions/73424263/why-do-most-ci-cd-platforms-use-yaml-for-their-pipelines) but was unable to get an answer from anyone.

If you know of a reason please do let me know in the comments.

## Solution

What I really think think would make things better is to use a langauge that is more flexible and better suited to the problem at hand.

If we need functions, variables and string interpolation why not use a language that has all of those things natively available?

You could use C#, Java, Go or any other language you like. The CI developer could build simple SDKs for each requested language.

Using my personal favourite language Typescript a simple build script could look something like:

```ts
import { exec, defineJob, persist, restore, defineWorkflow } from "ci";
import * as netlify from "netlify";

function doSomeCommonPreJobStuff() {
  console.log(`starting job..`);
}

const install = defineJob({
  label: `install`,
  description: `installs dependencies for the project`,
  handler: async () => {
    doSomeCommonPreJobStuff();
    await exec(`yarn install`, { label: "installs dependencies using yarn" });
    await persist([`./node_modules`]);
  },
});

const build = defineJob({
  label: `build`,
  description: `builds the project`,
  machine: `4x-16gb`,
  handler: async () => {
    doSomeCommonPreJobStuff();
    await restore();
    await exec(`yarn build`);
    await persist([`./dist`]);
  },
});

const test = defineJob({
  label: `test`,
  image: `cimg/node:16.15.1`,
  description: `tests the project`,
  handler: async () => {
    doSomeCommonPreJobStuff();
    await restore();
    await exec(`yarn test`);
  },
});

const deploy = defineJob({
  label: `deploy`,
  description: `deploys the project to netlify`,
  handler: async () => {
    doSomeCommonPreJobStuff();
    await restore();
    netlify.deploy(`./dist`, { apiToken: process.env.NETLIFY_TOKEN });
  },
});

defineWorkflow({
  label: `main workflow`,
  jobs: [install, [build, test], [deploy]],
});
```

It is almost reminiscent of a [gulp.js](https://gulpjs.com/) script if you are old enough to remember those.

It would be far more usable and easy to understand and critically testable..

# Testing & Iteration

## Problems

Another big problem with CI scripts is their development cycle is slow and painful and usually looks something like:

1. Write a few lines of YAML, hope that you have the syntax and commands correct and havent gotten a single whitespace of indentation wrong somewhere.
2. Git commit and push to the CI
3. Alt-tab to browser, wait for the CI to pick up the job
4. Usually errors at this point because of syntax issue in YAML, return to step 1. to get that right first
5. Wait for the install dependencies step to run (minutes)
6. Finally get to the bit you just added and discover it has errored for some reason, not sure why, return to step 1 to try some other minor change

In this day and age why oh why do we have to do things like this?

## Solution

We solved the problem of local execution of untrusted code some time ago with Virtual Machines and Docker, why cant we use those?

Why cant we use something like [Firecracker VMs as Fly.io does](https://fly.io/docs/reference/architecture/#microvms), that way you can execute docker instances with them without any problems. They also support snapshots similar to docker so you could imagine some interesting replay and rewind UI / UX scenarios this would empower. Im thinking something like [Cypress](https://www.cypress.io/) but for CI scripts.

[![](./cypress.png)](./cypress.png)

Another bonus of writing CI scripts in a general purpose language like Typescript we could easily unit test them, bringing another level of confidence that complex scripts work and will continue to work.

# Infrastructure as Code

IaC is a powerful technology that makes the infrastructure that runs your applications repeatable and manageable.

For the longest time it too suffered from the same problems as CI scripts, too much declarative code in the form of [Cloud Formation](https://aws.amazon.com/cloudformation/) or [ARM](https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview#template-deployment) Templates but then came along tools such as [CDK](https://docs.aws.amazon.com/cdk/v2/guide/home.html) or [Pulumi](https://www.pulumi.com/) which replaced declarative nightmare with a far more intuitive scripting syntax.

# Conclusion

We can do better, we have the technology, so why dont we? I dont know.

Maybe I have missed something, perhaps there is a very good reason why its done this way. Or perhaps there is a project or solution that always works exactly as I describe it.

If this is the case please leave me a comment below or email me if it is the case! If its not the case, IMO this would be an excellent SaaS business for someone brave enough to take it on.


2023

Archive

2023

Archive

2023

Archive