Back in in the before-times of May 2020 OpenAI made a [stunning announcement](https://openai.com/blog/tags/announcements/) about the capabilities of their latest project named GPT3. When I saw [what it was capable of](https://beta.openai.com/) and it was going to be turned into an API I knew I was going to have to tinker with it!



# TL;DR

I tried out the worlds most advanced Natural Language Processing AI Model on a real world problem and received mixed results.

# GPT3

Generative Pre-trained Transformer 3 (GPT-3) GPT3 is a MASSIVE (175 billion parameter) Natural Langue model that was trained on basically the entire internet. This means that given any chunk of text its able to comprehend it and generate a realistic output.

Seriously, if you haven't seen some of the things this AI can do stop reading this now and head over to https://beta.openai.com/examples and check them out, its pretty mind blowing.

![](./examples1.png)

The power and possibilities of this tech are endless..

.. that is if it works as well as the examples indicate _portentous music plays_.

Eager to try it out for myself, I signed up for the closed beta on the spot.

# Many Months Later

I finally received my invite to try it out. My first step was to play with some of the examples in their playground.

<iframe width="853" height="480" src="https://www.youtube.com/embed/iumG64XEBik" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

So far so good, looks awesome!

Then I decided to set it a little challenge. One of the examples is called "TL;DR summarization":

> This prompt summarizes text by adding a 'tl;dr:' to the end of a text passage. It shows that the API understands how to perform a number of tasks with no instructions.

So given this post from my blog: [Introducing BattleTabs](https://mikecann.blog/posts/introducing-battletabs), I wonder what kind of tl;dr it will give me..

![](./tldr1.png)

Ah that doesn't look right. Perhaps its struggling because I gave it Markdown rather than raw text?

I decided to remove all the Markdown markup and run it again..

![](./tldr2.png)

Awesome! It now works and gives me a really nice summary :)

# Whats Next?

I wondered what real-world problem I could give it that would be simple but interesting enough to test its capabilities a little more.

I played with the examples a bit more while I pondered.

[One caught my eye](https://beta.openai.com/examples/default-keywords):

> Extract keywords from a block of text. At a lower temperature it picks keywords from the text. At a higher temperature it will generate related keywords which can be helpful for creating search indexes.

Well, this blog has over 600 posts on it and many of those posts are old and are missing Tags. Could I run all my tagless posts through GPT-3 and use this keyword extractor to generate the tags, I wondered.

It might be worth a shot, it would save me a bunch of time manually going through them and improve the navigability of this blog in the process.

First step was to try out this keyword extractor with my manually-markdown-stripped text from before.

As "[Introducing BattleTabs](https://mikecann.blog/posts/introducing-battletabs)" is a fairly recent post it already has some manually hand-crafted tags:

```
- chrome extension
- game
- async
- typescript
- battleships
```

Lets see what GPT-3 comes up with:

![](./keywords1.png)

Wow! Impressive, sure its not quite as good as my manually crafted tags but still it worked really well.

Confident with my little experiment I set to work writing a script that would run through all my tagless posts strip out the text from the markdown and call the OpenAI API to get the keywords.

Simples! ...

It turns out that stripping the markdown markup from text is actually more complex that I thought it would be. To cut a long story short I decided to render the Markdown to HTML then using Puppeteer I extract the text from the page using `element.textContent`.

I will spare you all the other steps but if you are interested you can checkout the final script [HERE](https://github.com/mikecann/mikecann.blog/blob/master/scripts/openAIKeywords.ts).

# Results

I ran the script and 623 seconds later it had run through the 213 tag-less posts from a total of 612 posts.

_If you are interested, I have uploaded all the results [HERE](https://gist.github.com/mikecann/6efd37e58c5b0a38e0182bf40ac6a71e)._

It was able to generate tags for 177 of those 213 posts leaving 39 left over that it couldnt find keywords for. This isnt too bad, I was expecting a bunch of the older posts to be too short or to reference the visual (image or video) content too much to be able to extract any meaningful keywords from the text.

But what about the ones it could find keywords for?

Well firstly the good news, it seems like it was able to find good keywords for some of them. For example [my post titled "Village Life" from back in 2012](https://mikecann.blog/posts/village-life) it was able to generate these tags:

```
"mobile",
"game",
"facebook",
"multiplayer",
"flash",
"social",
"facebook",
"multiplayer",
"flash",
"social",
"facebook",
"multiplayer",
"flash",
"social",
"facebook",
"multiplayer",
"flash",
"social",
"facebook",
"multiplayer",
"flash",
"social",
"facebook",
"multiplayer,"
```

If you take the first 6 tags `mobile, game, facebook, multiplayer, flash` which is pretty impressive given the entire of the blog post text is:

```
Village life is a flash-based social game I worked on while at Playdemic. Its incredibly popular and has been played by millions of people.

I was responsible for the underlying client side technology that was used for developing the game.

Play it on facebook: https://apps.facebook.com/villagelifegame
```

Notice that nowhere in the post do I mention the word "multiplayer" or "social" but somehow it was able recognize that the game is that its "multiplayer" and "social" too.

I suspect it knows this from the various news stories about that game around the web that it gobbled up during its training. This would also explain the "mobile" tag and segway us into the negatives results of this experiment.

The problem with the "mobile" tag is that even tho the game was eventually ported to mobile I wasn't involved with the port and at the time the blog post was written the port didn't exist for mobile.

So this is a bit of a problem if I want to use these automatically generated tags for my posts as im now labelling my posts with falsehoods.

The issues continue. You many have noticed that the tags it generated are repeated many times. Well this seems to occur more often than not e.g.

```
 {
    "title": "Started Work at TheBroth in Perth!",
    "prompt": "Blog Post: Just a quick update. After a year of living down in Bunbury Western Australia and working Freelance and on my own projects (see this blog for more) I have decided to re-enter the workforce. Mainly because I was going a little stir crazy being by myself most of the day every day.\n\n\nI have taken a job 2 hours north in Perth at a company called TheBroth. We mainly work on a mobile and web game called SlotsGalaxy. Its similar sort of work to which I was doing at Playdemic so hopefully my old skills arent too rusty :)\nIm going to continue to work on my projects such as Mr Nibbles Forever and Post To Tumblr and my (still) unreleased strategy game but obviously time is going to be a little more limited.\nI have enjoyed the past year mostly but I think it was time for a change and im looking forward to the next year!\n\nKeywords: ",
    "tags": [
      "games",
      "game design",
      "game development blog",
      "game design blog",
      "games blog",
      "game development blog",
      "game design blog",
      "games blog",
      "game development blog",
      "game design blog",
      "games blog",
      "game development blog",
      "game design blog",
      "games"
    ]
  }
```

or worse it just gives up and starts repeating the same keyword:

```
{
    "title": "Supreme Commander Wallpaper",
    "prompt": "Blog Post: Well i have just finished making a massive (4608x2048) duel-screen wallpaper for a thread on the supreme commander forums.\n(click for full screen)\n\nKeywords: ",
    "tags": [
      "for alliance",
      "for allianc",
      "for allianc",
      "for alliance",
      "for allianc",
      "for allianc",
      "for allianc",
      "for allianc",
      "for"
    ]
  },
```

This could obviously be mitigated by removing duplicates but its worrying as the even the de-duplicated tags arent really that useful.

For example for the above Supreme Commander post the de-duplicated tags would be: `for alliance, for allianc, for` which doesnt actually say anything about the post.

# Whats Going On?

Why does the result of that original experiment on the "Introducing BattleTabs" post come out so good but when I run my script many of the results come out garbage?

I decided to run a quick text, take the Supreme Commander post and manually put it into the OpenAI Playground and see what I get:

![](./supcom1.png)

Well thats not perfect, I would expect it to mention something about wallpaper or perhaps creation, Photoshop or something but at least its not repeating and seems to be more relevant to the game.

# Conclusion

Unfortunately I have run out of time to tinker on this little side-quest at the moment.

My conclusion is that although very impressive, the purpose I used the model for is not quite what I was looking for so in the end I decided not to use the generated tags.

Im 100% sure that this is the future tho. I mentioned [back in 2017](https://mikecann.blog/posts/a-game-developer-learns-machine-learning-intent) that:

> I see a future where an AI is capable of generating an entire game based on your preferences such as "I would like a game that plays like Mario but has the same size and scope as World of Warcraft", the AI then generates variants on that for you to play.

GPT-3 is definitely leading the way towards that being a possibility. If you dont believe me checkout [this tweet](https://twitter.com/sharifshameem/status/1282676454690451457?s=20) of someone using GPT-3 to generate HTML from some natural language.

So, although for my example I don't think its quite there, I can definitely see this becoming more and more a part of our lives.

I see programming changing drastically over the coming years, going from less of telling the computer how to do something and more having a conversation with the computer and asking it to iterate on what it generated. If you want a glimpse at what this awesome future may be like then checkout the incredible [GitHub Copilot](https://copilot.github.com/).

P.S. The generated keywords for this post are: `AI, NLP, NLU, Natural Language Understanding, Machine Learning` :)

Im not sure how I first came across [Dfinity](dfinity.org) and the Internet Computer project but when I heard about the technologies they were using (Blockchain, Actors, Motoko) and the goals they were striving for I knew I had to have a tinker.



# Background

So what is the Internet Computer? Well ill quote [from their website](https://dfinity.org/):

> The Internet Computer is a public blockchain network so powerful that it can natively host hyperscale open internet services, pan-industry platforms, DeFi systems, secure enterprise systems, websites, and all of humanity's software logic and data in smart contracts. It is created by independent data centers worldwide running the Internet Computer Protocol (ICP).

Theres a lot of buzzwords in there but I see it as a decentralized app execution environment much like Heroku or parts of AWS or Azure. Much like those services you dont have to worry about the hardware that runs your code, you just deploy your code and it begins to execute.

[There a bunch of other cool things](https://dfinity.org/faq/) that make the project exciting from the way that the code executes in a "tamperproof" manner to the way the blockchain and tokens are used to both pay for execution and to operate as a governance mechanism.

As a programmer and tech nerd the main thing that interested me is their new programming language the way applications are structured in the form of the Actor Model.

So lets tinker..

# Getting Setup

Unfortunately the IC's SDK currently only runs on Linux or Mac so to get it to work on Windows I first had to install the Windows Subsystem for Linux 2 which wasn't too bad thanks to [this guide](https://www.omgubuntu.co.uk/how-to-install-wsl2-on-windows-10) but still a little frustrating.

Once setup I was able to install the SDK and create my first project using their command line.

![](./ic-terminal1.png)

I followed their [quick start guide](https://sdk.dfinity.org/docs/quickstart/local-quickstart.html) and soon had my first little application running.

![](./hello-world.png)

As you can see from the image above, your application is split into two parts; the "src/my_app_assets" is the frontend and is your typical Webpack produced Single Page Application and the backend "src/my_app" which is the interesting part in the form of the Motoko actor.

# Actors

Functions you define to run on the Internet Computer run inside something called an Actor. Actors are an [old but fascinating](https://www.brianstorti.com/the-actor-model/#:~:text=The%20actor%20model%20is%20a,this%20model%20is%20probably%20Erlang%20.) concept and are used projects such as [AKKA](https://akka.io/) and [Project Orleans](https://dotnet.github.io/orleans/).

The basic concept is that code and state are located together much like classes in typical OO programming. On the IC actors are written using their special purpose langauge [Motoko](https://sdk.dfinity.org/docs/language-guide/motoko.html), heres an example:

```motoko
actor Counter {

  var value = 0;

  public func inc() : async Nat {
    value += 1;
    return value;
  };
}
```

As you can see you simply define an actor `Counter` which has some state `value` which can be incremented by calling `inc` and then returned to the caller.

Actors dont need to worry about saving their state to a database, the state is automatically persisted and depersisted by the system as actors are invoked.

The beauty of this model is its simplicity. You no longer have to worry about how to structure your DB tables and indices, simply pass a message to an actor (calling a function) to query or mutate its data.

Calling an actor from the frontend is as simple as importing it and calling it:

```js
import my_app from "ic:canisters/my_app";

my_app.greet(window.prompt("Enter your name:")).then((greeting) => {
  window.alert(greeting);
});
```

And because its just an RPC the typescript definitions for your actor are easily generated.

# Queries vs Mutation

The Motoko language has two types of functions you can define in your actor; "query" functions only allow you to read and manipulate data from the actor and "mutation" (my term not theirs) functions that also allow you to update the state of the actor.

The reason why they have two kinds of functions is a bit complicated but has to do with consensus in a decentralized system. When you query data you dont need consensus, the response can simply be retrieved and returned but for a state change (mutation) to happen it must be agreed upon by multiple nodes. This consensus between nodes in the IC takes time and thus there is a delay.

This delay time is actually quite significant, the docs quote [1-2 seconds](https://dfinity.org/faq/), but when running locally its more like 3+ seconds. This might not sound like a lot but when you compare it to the millisecond responses everyone is used to with modern websites it does highlight a bit of an issue.

This inherent mutation delay sounds like a bit of a deal-breaker but if you are careful in the way you apply optimistic state updates on the client, [you can make it appear as if the state change happened immediately](https://medium.com/dfinity/how-i-built-a-multiplayer-reversi-game-on-the-internet-computer-f67d2fed0fc3).

# Other Problems

So there is a lot to be excited about with the IC however I would be remiss if I didn't mention a few issues that I encountered on my tinkering journey.

## No Unit Testing

So far the docs don't mention any sort of automated testing capabilities. I assume that this is something that they will implement at some point but IMO its really important to have this baked in from day one particularly if you are building a Blockchain / DeFi application.

## No Intellisense

Even though there is syntax highlighting and a VSCode plugin there is unfortunately no intellisense or other nice features such as refactoring that im used to from other languages. Again its early days for the project so im sure that will come in time.

## Slow deploy / development iteration process

When you make a change to your Motoko file you must then run `dfx deploy` to build and deploy to your locally running environment. Unfortunately this process currently takes a long time. On my powerful desktop computer it takes 19 seconds each time.

Again im hoping this will be fixed as the project progresses more. I think unit tests could help greatly here too as you can be more confident that your code is going to work before you hit "deploy".

## Not Open Source

This has been a bit of a sore point for members of the blockchain community who are used to cypto projects being open source from the beginning. Dfinity have promised to open source everything at somepoint however.

## Okay-ish docs

I think this is the nature of an early, rapidly developing project but it seems like chunks of the docs are out of date which leads to a bit of confusion. For example some of the tutorials recommend using `dfx deploy` whereas others talk about using multiple deploy steps such as `dfx canister create --all` then `dfx build` then `dfx canister install` etc.

## Not Yet Live

You can currently push code to a "testnet" which will allow you to run your code remotely and have others access it but this is not considered production ready. There are still lots of questions around what happens when the code is in production, who can view the code and data etc.

# Conclusion

I have barely scratched the surface of the Internet Computer but what I have seen I am very impressed with.

I think for my next step I would like to build something a little more meaningful and have it running on a remote network so others can access it.

Right now however im going to keep an eye on the project and wait a while. They have a bit more work to do to make it user friendly and I don't want to pay the early adopter tax when there are so many other cool shiny things to play with