Please fill out as many of the following questions as possible so we can help you promptly! If you don’t know how to answer or it does not apply, feel free to remove it or leave it blank.
Builder content link
N/A
Builder public api key
e7d73274a70543c4bb1fdf4d0c80ce77
What are you trying to accomplish
I want to have some routes defined within my code-base and some routes defined in builder. Currently, only routes defined in builder work while locally defined routes are sent to 404 page. This does not happen in the dev environment. It is only happening in production.
The locally defined routes are visible if the user clears their cache, but as this application has thousands of users, it is not feasible to ask each one to do so.
Code stack you are integrating Builder with
NextJS @13.0.1
Browsers
Issue seen on Safari, Firefox, Chrome and Brave
Could you please share some example routes defined in the builder, URLs defined in the code-base and how you are handling them currently from NextJS? Also, can you confirm the NextJS version you are using on the dev environment?
The routes defined in NextJS work inconsistently, but all of the routes defined in NextJS are the ones that experience this issue. For example, clearing the cache once solved the issue for me, while someone else on my team experiences this every time. Regardless, no one visiting the site should have to clear their cache.
NextJS routing example:
/Pages
/auth
sign-in.tsx - page content defined here
We tried to test the sign-in page that is sent and it works fine every time for us, what might help us get to the bottom of the issue, is if you could share the code for one of the pages where the route doesn’t work. Thank you!
Seconds to cache content. This sets the maximum age of the cache-control header response header. Set the value higher for better performance, and lower for content that changes frequently.
For issue on NextJS cache issue, You can set the Cache-Control header in your Next.js API Routes by using the res.setHeader method or you can use caching headers inside getServerSideProps and API Routes for dynamic responses. For example, using stale-while-revalidate.
// This value is considered fresh for ten seconds (s-maxage=10).
// If a request is repeated within the next 10 seconds, the previously
// cached value will still be fresh. If the request is repeated before 59 seconds,
// the cached value will be stale but still render (stale-while-revalidate=59).
//
// In the background, a revalidation request will be made to populate the cache
// with a fresh value. If you refresh the page, you will see the new value.
export async function getServerSideProps({ req, res }) {
res.setHeader(
'Cache-Control',
'public, s-maxage=10, stale-while-revalidate=59'
)
return {
props: {},
}
}
By default, Cache-Control headers will be set differently depending on how your page fetches data.
If the page uses getServerSideProps or getInitialProps, it will use the default Cache-Control header set by next start in order to prevent accidental caching of responses that cannot be cached. If you want a different cache behavior while using getServerSideProps, use res.setHeader('Cache-Control', 'value_you_prefer') inside of the function as shown in the below documentation.
If the page is using getStaticProps, it will have a Cache-Control header of s-maxage=REVALIDATE_SECONDS, stale-while-revalidate, or if revalidate is not used, s-maxage=31536000, stale-while-revalidate to cache for the maximum age possible.
After investigating the issue further, I have pinpointed the issue. I initially launched with builder-only pages. On our second release, I added a few non-builder pages. It appears that users who accessed our site before the second release experienced this issue. From my perspective, this can only be an issue with builder assets being cached for too long resulting in non-builder pages being sent to 404.
I do not want to stop caching other static assets solely to solve this issue. Is there a way we can force only the builder assets to update? Here is what I’ve tried recently, but it hasn’t resolved the issue for the users affected.
export async function getStaticPaths() {
const pages = await builder.getAll('page', {
options: { noTargeting: true },
omit: 'data.blocks',
// added the following 2 lines
cache: false, // should prevent caching in the future?
cachebust: true, // should clear any existing cache?
});
return {
paths: pages.map((page) => `${page.data?.url}`),
fallback: true,
};
}
You can try using Seconds to cache content. This sets the maximum age of the cache-control header response header. A set value is higher for better performance, and lower for content that changes frequently.
We use a technique called stale-while-revalidating caching which updates the data based on how frequently data is requested. For example, if you update the content, but no one visits your site, your content may not update right away. The next time a user visits the site, they’ll see the cached version, while the latest version is fetched in the background and the cache is updated. The next visitor (after a few seconds) will see the new version.
You can control the cache length via query params in the request to our content api . We recommend keeping the default values though, since reducing cache time can negatively affect performance. Generally speaking, a development site (that is not live) could be a little stale, but a live site with real traffic will always be fast and fresh. If you want to manually ensure pages are updated, but you have no one on your website visiting them, you can always just visit them once or twice after updating to ensure we fetch the latest version.
The site does receive traffic, but defining what “real” is the question. We have had 100s of users experience this issue that we know of. And the thing that fixes it for each person is clearing their cache. This indicates to me that it is each user’s local cache that is causing the issue. I’m looking for a solution to clear that existing local cache.
Clearing the cache for some users is only a temporary fix. Here is the workflow:
User navigates to sign in page
404 page displayed
User clears their browser cache
User is now able to successfully navigate to sign in page
Next day, user navigates to sign in page and is shown the 404 page again
This user journey applies to all non-builder pages that have been set up. This indicates that the content from builder is being cached on your end and not updating.
When you navigate to sign-in page, what is being served from the cache? And where is it cached?
Builder does not control your nextjs routing other than the catch-all for builder routes
Can you set up a repro case if you still want to look into this
If you could also share the code snippets on how you have set up the routing would be much appreciated. Earlier the code you shared was for the page component which doesn’t help us much in case of a routing cache issue. Thank you!
What is being served from the cache? And where is it cached? - This I am unclear of. I know that for every user experiencing the issue, clearing their browser cache momentarily fixes this issue.
Builder does not control your nextjs routing other than the catch-all for builder routes - this I understand, but it is rendering the error component because it does not recognize locally defined pages.
Can you set up a repro case if you still want to look into this - what is a repro case?
Code snippets/routing - routing in Next.js is defined by the directory structure instead of in a file like in regular React. So the example I provided before would be in the /pages/auth/sign-in.tsx. The only code related to routing is almost entirely copied from builder docs. Here are the contents of [[…page]].tsx
import type { GetStaticPropsContext, InferGetStaticPropsType } from 'next';
import { useRouter } from 'next/router';
import { BuilderComponent, builder, useIsPreviewing } from '@builder.io/react';
import Head from 'next/head';
import ErrorPage from 'src/components/ErrorPage/ErrorPage';
import builderConfig from 'src/config/builder';
builder.init(builderConfig.apiKey);
// import any components needed for builder.io
import 'src/components/ImageCarousel/ImageCarousel';
import 'src/components/PropertySearch/PropertySearch';
import 'src/components/builder.io/FamilyHero/FamilyHero';
import 'src/components/builder.io/LearnMore/LearnMore';
import 'src/components/builder.io/PageHeader/PageHeader';
import 'src/components/builder.io/Highlights/Highlight/Highlight';
import 'src/components/builder.io/Highlights/HighlightRow/HighlightRow';
import 'src/components/builder.io/FAQs/FAQ/FAQ';
import 'src/components/builder.io/FAQs/FAQWrapper/FAQWrapper';
import 'src/components/builder.io/Founders/Founder/Founder';
import 'src/components/builder.io/Founders/FounderWrapper/FounderWrapper';
import 'src/components/builder.io/Marketing/MarketingStep/MarketingStep';
import 'src/components/builder.io/Marketing/MarketingSteps/MarketingSteps';
import 'src/components/builder.io/GenericPage/GenericPage';
import 'src/components/builder.io/CountyHeader/CountyHeader';
export async function getStaticProps({ params }: GetStaticPropsContext<{ page: string[] }>) {
const page =
(await builder
.get('page', {
userAttributes: {
urlPath: '/' + (params?.page?.join('/') || ''),
},
})
.toPromise()) || null;
return {
props: {
page,
},
// Next.js will attempt to re-generate the page:
// - When a request comes in
// - At most once every 5 seconds
revalidate: 5,
};
}
export async function getStaticPaths() {
const pages = await builder.getAll('page', {
options: { noTargeting: true },
omit: 'data.blocks',
cache: false, // should prevent caching in the future?
cachebust: true, // should clear any existing cache?
});
return {
paths: pages.map((page) => `${page.data?.url}`),
fallback: true,
};
}
export default function Page({ page }: InferGetStaticPropsType<typeof getStaticProps>) {
const router = useRouter();
const isPreviewingInBuilder = useIsPreviewing();
const show404 = router.isFallback || (!page && !isPreviewingInBuilder);
return (
<>
<Head>
<title>{page?.data.title}</title>
<meta name="viewport" content="width=device-width, initial-scale=1" />
{!page && <meta name="robots" content="noindex" />}
</Head>
<main>{show404 ? <ErrorPage /> : <BuilderComponent model="page" content={page} />}</main>
</>
);
}
Unfortunately, this code and logs didn’t help much in reproducing this possible issue that you are facing with non-builder pages only, you might need to try and reproduce this from your end, try and create a new nextJS sample project and include one builder page route and one from nextJS, and then see if you can reproduce this possible issue. Let us know how that works for you.