How I extend my blog with gamified learning
One of the major reason that I write all of these blog posts is to help people learn about cloud and AWS. How would you know that you understood what you read and learned from it? In this post I discuss how I introduced gamified learning by adding a quiz from kvist.ai on my blog posts, directly from my CI/CD pipeline running as an event-driven system using Amazon EventBridge and AWS StepFunctions.
Published Jul 25, 2024
One of the major reason that I write all of these blog posts is to help people learn about cloud and AWS. How would you know that you understood what you read and learned from it? I got a suggestion to add a quiz to the end of the blog, that was based on the content of the blog. This was great suggestion and I got to work on it.
kvist.ai is a Generative AI powered quiz solution powered by AWS, Serverless technologies, and Amazon Bedrock. Feeding this solution with my blog it automatically creates a quiz based on the content, which is exactly what I needed. Now, knowing the Founder, Lars Jacobsson, I did get access to some early features that helped my automate the process.
In this post I will discuss how I extended my CI/CD solution to add extra step to create the quiz and add it to the blog. My pipeline run in AWS and is serverless, event-driven, and powered by AWS StepFunctions, Lambda, EventBridge, and more.
To make sure you all understand how my blog is created and distributed, this will make it easier to understand the setup later. My blog consists of pure old static HTML, no React, Vue, or anything like that. The blog is distributed over CloudFront and S3, I use Lambda@Edge and CloudFront functions to manipulate the response and collect statistics, check out my post "Serverless statistics solution with Lambda@Edge" on that matter.
I write and create my posts using markdown, this is then converted to html with 11ty engine. The layout of the page are decided by the metadata in the front matter section, 11ty the uses the layouts I have created using Nunjucks. This way I can add metadata and control how the page is rendered, I can inject sections and links.
My build and deploy pipeline is based on GitHub and GitHub actions, which build the site on merge to main branch and sync the html files to the S3 bucket.
My CI/CD setup consists of two major parts, the build and deploy part that is invoked on a merge to main branch. This part is 100% GitHub actions based and looks like the part above.
The second part is where I perform two major things, I generate the quiz for the gamified learning and I use Polly to generate voice that reads my blog post. This is a mix of GitHub actions, that will build the page and upload to a staging bucket in S3, this part is invoked when I open, close, and modify a pull-request. After upload to the staging bucket is complete, GitHub actions will post an event onto an EventBridge event-bus and this is where my AWS based part takes over. In this blog we will focus on this AWS based part.
The AWS based CI/CD pipeline is event-driven and serverless, the primary services used are StepFunctions, Lambda and EventBridge. The flow is based on a saga pattern where the domain services hand over by post domain events on the event-bus, which will move the saga to the next step.
To summarize the image a bit, GitHub action will invoke the Information service that will reach out to GitHub to collect information about the pull-request, this will then invoke the Voice service that will generate the Polly text to speech action, followed by the Quiz service creating a quiz on Kvist.ai, finally the update service will edit the front matter of the markdown file and create a new commit on the pull-request branch.
Now, why do I implement this pipeline in AWS with StepFunctions and Lambda? Why not just implement it in GitHub actions. The answer is that I could do that, but I need to repeatedly call different Service API in AWS and with the integration with kvist.ai now it was just easier to implement it in AWS directly.
We have now come to the most fun part of this post, the technical deep dive. In this part I will try and explain and show how each of the services work. If we creates the overview image again but now with more details and AWS services, it will look like this.
There are still a lot going on in this image, but don't worry we will go into each of the services one by one and discuss the architecture and data flow.
First let's take a quick look at the event structure, so you get an understanding how data are added. I have opted in to use the metadata-data pattern where each service will add information and post a new event onto the bus, so the next service in the saga can perform its part.
In the end an event looking like this will be posted, it has all the information needed for the update service to create a new commit.
Normally you would not keep data from the invoking event, but since the entire chain is built as an saga pattern and instead of each service fetching the same information over and over again, it was easier to break the rules a bit.
The very first step is to collect information about the pull-request. I will fetch what markdown file that was updated, where the rendered html file can be found in the staging bucket, what branch the pull-request originated from. This is done with a couple of Lambda functions running in sequence in a StepFunction.
To communicate with GitHub I use Octokit. Below are snippets from the CloudFormation Template and the code used to call GitHub API.
The logic in this first part is not that complex, it's most about fetching the correct information.
The most complex part of this entire setup is the voice generation with Polly. I will start with an
extract
Lambda function, that will fetch the HTML file that will be use. In the transform
part I create the SSML used by Polly when reading the post.Now, I will not go into details on this part as its actually a blog of its own. To learn more about it check out my post Serverless voice with Amazon Polly or watch my conversation with Johannes Koch on YouTube
Now it's time to generate the quiz. This is done by calling an API provided by kvist.ai, this is called using an HTTP Endpoint task in StepFunctions. This is a great way to call an API without having to write any form of code.
To create a HTTP EndPoint you use the
http:invoke
state, below is a snippet from the asl file for the state machine.Looking in Workflow Studie we will end up with a configuration like this.
For a deeper dive into StepFunctions HTTP EndPoint I recommend my post AWS StepFunctions HTTP Endpoint demystified
The final step in this saga is to update the markdown file, add the correct front matter, and the create a new commit.
In the first part of the StepFunction I copy the mp3 file that Polly generated to the correct location, I will not check in the mp3 files in the repo as these ar huge blobs and can be reproduced. Instead I copy them to a S3 bucket and during the build process the files will be pulled from this bucket and placed into the production bucket.
When updating the front matter I use gray-matter to make it a easy process. The code will check if there is a voice and quiz section in the event, and if so add new tags in the front matter part.
This will now create a front matter section looking like this one.
The important parts here are
audio
and quiz
as these will be used to render out special sections in the blog post.The last part will create a new commit in the repo, this is once again done with Octokit, it will involve several steps as create the commit, upload files to the repo, etc. Below is a simplified version of the code.
As the entire CI/CD solution is built as an event-driven system, using the saga pattern, the extension with a new step is very straight forward. This is where an event-driven solution really shine. I could develop the entire quiz service on the side and just invoke it on the same event as the voice service. When I was happy with the result, I updated the saga and changed what events invoked what parts.
The overall time to extend this solution was less than one hour. I didn't have to update any complex flows, just add a service and change the saga a bit. There is a reason why I truly enjoy working with serverless and event-driven systems.
As mentioned before I use 11TY to render the blog posts from markdown to html. During this process 11TY will use the layout I have created for blog posts.
In this layout I have sections
'% if audio %'
and '% if quiz %'
to check if this is available in the front matter. If it is it will add the html tags inside the if-blocks. Automatically adding audio and the quiz based on the data from the front matter.This was a post where I explain how I added a step in my event-driven CI/CD pipeline to create a gamified learning experience for you, my readers. If you enjoy the quiz part go and check out kvist.ai
Check out My serverless Handbook for some of the concepts mentioned in this post.
As Werner says! Now Go Build!