
Optimizing AWS AppRunner applications for SEO
Tips and tricks to make AWS AppRunner SEO proof
- My code is hosted on GitHub in a monorepo
- I use GitHub Actions to build my app. When I push to the main branch of my repo, I want my new app to be deloyed automatically to production.
- I have a dockerized NextJS application for Server Side Rendering
- I am a solopreneur! I don't want to spend too much time managing and scaling infrastructure or devops pipelines.
- all content must be served via HTTPS, but I don't want to manage SSL certificates myself
- I'm expecting thousands of users a month, so the app needs to scale, and I don't want to deal with scaling policies or autoscaling groups
- I need a custom domain
https://myapp.eu-west-1.awsapprunner.com
robots.txt
. Robots.txt is a simple text file used in web development and SEO to control how search engines crawl and index a website. By specifying directives, you can instruct search engine bots on which pages or paths to include or exclude from indexing. This helps manage your site's SEO by ensuring that only the most important content is indexed, preventing the crawling of duplicate or irrelevant pages, and protecting sensitive areas of your site from being publicly accessible in search results.The trick here is to use the host
header of the HTTP request and render a response conditionally.In ExpressJS you would do something like this
1
2
3
4
5
6
7
8
9
10
app.get('/robots.txt', function (req, res) {
// we're expecting req.headers.host === "myapp.eu-west-1.awsapprunner.com."
// when a bot starts crawling the non-custom domain
if (req.headers.host.split(".").includes("awsapprunner")) {
res.type('text/plain');
res.send("User-agent: *\nDisallow: /");
}else{
// render the default robots.txt
}
});
*.awsapprunner.com
. But, there's a but! Nothing will stop a clever user to spoof my apprunner domain and use that url. Not a great user experience. You might still want to use this option to disable indexing of your dev and test environments for example.1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import { NextResponse } from 'next/server';
export function middleware(request) {
const currentHostname = request.headers.get('host');
// Check if the request is already for desired domain
if (currentHostname === 'accento.ai') {
// No redirection needed, continue processing
return NextResponse.next();
}
const url = request.nextUrl.clone();
url.pathname = request.nextUrl.pathname === '/' ? '/' : request.nextUrl.pathname; // Preserve path if not root
url.hostname = 'accento.ai';
url.port = '';
// we want the redirect to be permanent
// more info : https://en.wikipedia.org/wiki/HTTP_301
return NextResponse.redirect(url, 301);
}
export const config = {
matcher: ['/((?!api|_next/static|_next/image|favicon.ico).*)'], // Match all except certain paths
};