Optimizing AWS AppRunner applications for SEO

I was looking for a quick, easy to manage way to deploy and scale a front-end application for my new startup. Chatting with an AWS Solutions Architect, we landed on AWS AppRunner.Here's a breakdown of my requirements:

My code is hosted on GitHub in a monorepo
I use GitHub Actions to build my app. When I push to the main branch of my repo, I want my new app to be deloyed automatically to production.
I have a dockerized NextJS application for Server Side Rendering
I am a solopreneur! I don't want to spend too much time managing and scaling infrastructure or devops pipelines.
all content must be served via HTTPS, but I don't want to manage SSL certificates myself
I'm expecting thousands of users a month, so the app needs to scale, and I don't want to deal with scaling policies or autoscaling groups
I need a custom domain

Now AppRunner is ticking all these boxes for me, except for a couple of important details.

Enter the SEO Requirements

A common challenge in SEO is making sure you don't have duplicate content indexed by search engines. When search engines encounter the same content in multiple places, they struggle to determine which version should rank higher. This can negatively impact your position in search results. Plus, it can weaken your site's overall authority as ranking signals may be split between duplicates. It can also lead to a poor user experience, where visitors might find the same info repeatedly, driving them away.

Why is this a problem?

When you publish an app on AppRunner, you'll be provided with a public URL in the following form

https://myapp.eu-west-1.awsapprunner.com

That's great for development and testing, but not so much for production! The problem is that with my default SEO configuration, the content I publish on my website will be indexed for both my custom domain—say accento.ai—and for *myapp.eu-west-1.awsapprunner.com .*Turns out you cannot disable the non-custom domain, so I was stuck with it. Or at least my friendly neighborhood Solutions Architect and I couldn't find a way. If you know a way let me know in the comments!As a result, my content is at risk of duplicate indexing, overall lowering my SEO score.

Solution 1: serving robots.txt dynamically

One option is to conditionally serving robots.txt . Robots.txt is a simple text file used in web development and SEO to control how search engines crawl and index a website. By specifying directives, you can instruct search engine bots on which pages or paths to include or exclude from indexing. This helps manage your site's SEO by ensuring that only the most important content is indexed, preventing the crawling of duplicate or irrelevant pages, and protecting sensitive areas of your site from being publicly accessible in search results.The trick here is to use the host header of the HTTP request and render a response conditionally.
In ExpressJS you would do something like this

1
2
3
4
5
6
7
8
9
10
app.get('/robots.txt', function (req, res) {
  // we're expecting req.headers.host === "myapp.eu-west-1.awsapprunner.com."
  // when a bot starts crawling the non-custom domain
  if (req.headers.host.split(".").includes("awsapprunner")) {
    res.type('text/plain');
    res.send("User-agent: *\nDisallow: /");
  }else{
    // render the default robots.txt
  }
});

This is simple and effective enough: it will stop bots from indexing content for *.awsapprunner.com. But, there's a but! Nothing will stop a clever user to spoof my apprunner domain and use that url. Not a great user experience. You might still want to use this option to disable indexing of your dev and test environments for example.

Solution 2: permanent redirect for all requests via NextJS middleware

This is a blanket solution that will permanently redirect all incoming requests for my non-custom domain to my custom domain. I use this on my production website, so that I don't need to worry about duplicate content and prevent my AppRunner non-custom domain from being used by bad actors. I make use of NextJs middleware to intercept every request, inspect the host header and redirect if needed. This way, bots will never index content under a domain different than my custom domain.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import { NextResponse } from 'next/server';
export function middleware(request) {
  const currentHostname = request.headers.get('host');
  
  // Check if the request is already for desired domain
  if (currentHostname === 'accento.ai') {
    // No redirection needed, continue processing
    return NextResponse.next(); 
  }
  const url = request.nextUrl.clone();
  url.pathname = request.nextUrl.pathname === '/' ? '/' : request.nextUrl.pathname; // Preserve path if not root
  url.hostname = 'accento.ai';
  url.port = '';
  // we want the redirect to be permanent
  // more info : https://en.wikipedia.org/wiki/HTTP_301
  return NextResponse.redirect(url, 301);
}
export const config = {
  matcher: ['/((?!api|_next/static|_next/image|favicon.ico).*)'], // Match all except certain paths
};

Conclusion

The sun shines again on AppRunner land! With a little bit of tinkering we managed to stop search engine bots crawling and indexing on AppRunner's default domain and avoid duplicate content hurting my SEO score.

About the Author

I'm Huzaifa, a 16-year-old Northern Pakistani solopreneur! I am passionate about creating micro-SaaS products and using AI to enhance productivity and efficiency. I'm working on Accento.ai to help you build a personal brand and automate the creation of LinkedIn content. Come say hi in the comments or on LinkedIn!

What's Accento.ai?

Accento.ai builds your brand and automates content creation. It's like having a personal assistant, minus the coffee runs. No more late-night writing marathons. No more staring at a blank screen. Just pure, high-quality content that actually works.And the best part? All this without hiring a pricey copywriter!Want to build your brand effortlessly? Give Accento.ai a try. Its top 3 features are bulk content creation, bulk content scheduling, and content re-purposing from blog posts and videos to engaging LinkedIn posts.

Acks!

I wanted to spend a few lines to thank the Showcase Slack community on Startups.aws! The AWS experts there helped me finding a workable solution for both my deployment setup and this SEO dilemma. If you're building a Startup on AWS, have a look at accento.ai's Showcase profile. Register for a free Showcase profile and you'll be invited to take part in the Slack community for founders and entrepreneurs.

Select your cookie preferences

Site Terms, Privacy, and more.