Hi @steve! Thanks for these notes. A few points:
- includeRefs=true in the API includes references (Rerence field type) values
As my testing above shows, it’s not so clear cut. includeRefs had no effect at all on resolving symbol input fields that were references. The only thing that mattered was noTraverse. I have noticed that includeRefs has an effect on top-level content item fields.
These kinds of discrepancies make the data flow confusing and give the impression that refs can’t be relied on. Knowing how Builder works, I think the issue is that the SDK and the content API logic aren’t always aligned on assumptions about user intent but the SDK depends on the content API. When there’s a misalignment, like you point out with the use case for includeRefs in the content API vs. the SDK, confusion ensues.
options={{}}in BuilderComponent is only applied if the component itself does the fetching, not if you fetch withbuilder.get()and pass content as a prop. the preferred way is usingbuilder.get(),we will be deprecating fetching within the react component (which include theoptionsprop) over time
I figured that something like this is true, thanks for confirming. I think what’s confusing here is that it’s not clear that <BuilderComponent> is something that would ever fetch content on its own, let alone under which circumstances. I’m still kinda fuzzy on this myself and will need to go back into the docs.
I’m glad to hear that it’s being deprecated. Sounds like the new approach is in line with the functional ergonomics of the v2 SDKs (fewer side effects === 
).
noTraverse=truetells the API to traverse the content to apply dynamic data like symbols, HTTP requests, custom server side js code, etc. at the API level it is off by default, but the SDKs sendnoTraverse=falseaka augment the data
Did you mean noTraverse=false?
Are you saying that noTraverse has implications beyond just symbols? If so, those should be documented. I’m not sure what the “etc.” in your response means precisely 
.
- the reason why API calls don’t include references or traversal by default is it is computationally expensive
I figured this was probably the reason, but in my experience there has never been a case where I don’t want all the refs for my content resolved. By not including the resolved refs by default, all that happens is that I have do the same work myself. In fact, it’s probably less efficient because now your servers have to process multiple network requests from my back end.
Once this PR is merged, your SDK will have a utility function to do exactly what I’m describing: https://github.com/BuilderIO/builder/pull/1238. So why not save the hassle and just turn it on by default?
(OK, having re-read your last bullet point, I get it. Point taken.)
Also, seems like this is an ideal use case for a revamped GraphQL API that could auto-resolve and only fetch whatever fields I tell it to.
as well as problematic if you are doing reads and writes in the API (you don’t want to write back to us with refs/symbols/etc included)
Isn’t the write API separate? The changes I’m advocating for would only affect the read API.
Also, why not let the user write back to you with refs/symbols included? I was basically trying to do exactly that here, only manually inside the Visual Editor’s JSON view: Reference field on a "repeat for each" symbol
most use cases for using the content APIs directly is fetching large lists of content to either modify and write back, or to scan for basic thing like “give me all of our Builder page URLs” which don’t need any of these refs/traversal. the SDKs are most commonly used for fetching single entries at a time, and need all refs and traversal, so those default to sending noTraverse=false (and soon will default to includeRefs=true too)
OK! This clarifies many things.
I may be wrong, but I don’t think it’s documented that the SDK turns on certain options by default, like noTraverse=false. I myself didn’t know that until now.
One of the annoying things about the current SDK docs is that you have to kinda guess the correct options by cross-referencing the content API doc section on query params with whatever IntelliSense pops up when typing builder.get(. Especially annoying that some things are at the top-level of the builder.get options object (e.g., { fields: 'data.url,data.title' }), while other things are implicitly nested within the undocumented options property (e.g., {options: { noTraverse: false }}). Or that something like fields expects a comma-separated list of params vs. something like sort that expects nested objects.
To get back to your original point, now that I think about it, probably the most common use case for content API calls is something like getStaticPaths in Next.js where you just want a list of all your content item URLs, which you’re right, has nothing to do with refs. The division of labor you outlined makes perfect sense: SDKs with resolved content, naked API requests without.
over time, the v3 API aims to consolidate these params, likely there will just be one includeRefs param that will do everything (traverse and apply refs, symbols, other dynamic data). this will be off by default but for SDKs will pass this to be on by default
Yes Yes Yes
also slotted for the v3 content API is recursive refs as well (so includeRefs=true will include all refs not just the first layer)
Ship It Ship It Ship It