Hi team,
I am investing on how to migrate our Old data to the builder.IO.
So far what I have learned is, to migrate the data, first I need to create data models in builder.IO and then use Builder.IO write API’s to post data of any data model.
But my concern is how can i migrate our millions of data into Builder.IO, for example we have millions of Blogs in our database that we want to migrate into Builder.IO.
Following are my questions:
-
The Write API eg: “https://builder.io/api/v1/write/blogs” can take only 1 record at a time, does that mean if i have to migrate my 1 milion of blog post to builder.IO then I have to hit Builder.IO Write API 1 million times?
-
Can Write API accept multiple records at a time?
-
Do you have any other tool which migrate our Old data to Builder.IO effectively.
-
Does Write API counts in the billing?
-
Can you provide or develop the new API which can accept multiple records at a time for us to migrate the data if we are in the enterprise plan?
Thanks
Sushil Suthar
Hi @sushil.suthar,
-
Yes, that’s right, If you have 1 million blog posts to migrate to builder.io then you will have to hit Write API 1 million times
-
Currently, write API can’t accept multiple records.
-
We do have an Admin CLI / SDK which customers can / have used before (builder/packages/admin-sdk at main · BuilderIO/builder · GitHub)
-
Write API requests that do not account for billing.
-
We might be able to, but we can’t promise, in past, we have always recommended just writing a script using the Write API to migrate large sets of data, but of course, you can also write a middleware on your own to balance all those requests.
Let us know if you have any further questions. Thank you!
Hi Manish,
Thanks for your response,
1 million hit, That’s a bad news for us, because this is very common requirement which builder.io doesn’t have, inserting multiple records in one API call.
If i take 1 second for 1 blog post to insert in builder.io for overall process, fetching from our database then making builder.IO write API call, overall if it takes 1 second then 1 million records will take around 11 days to complete, and that will be a very bad practice, even 0.5 seconds will also take 5 days.
Do you have any other approach to reduce the time?
Hi @sushil.suthar,
Currently, we don’t have any other approach but feel free to submit a feature request at Builder.io Ideas