Do NOT run a/b tests over builder! Tracking is completely incorrect

Bugs
1

Hi,

we compared builder.ios tracking with that of TripleWhale and Google Analytics and were shocked to find that builder only manages to report a fraction of actual sales. We stopped running a/b tests over builder since the tracking is so inaccurate that we cannot base business decisions on the results. Some examples:

November 4th - November 10th

Shopify: reports 854 orders overall, 628 of which are orders that came through the PDP that we built with builder.

Builder: Of that 628 orders that came through that builder PDP, builder only reports 260 conversions

Triplewhale: of that 854 orders that we had overall, triple whale manages to track 721

Google Analytics: Of the 854 orders overall, Google Analytics manages to track 706. Of that 628 orders that came through the builder.io PDP, Google analytics manages to track 519

This is just one week to illustrate the mismatch but this goes for every time range.

Builder content link

Code stack you are integrating Builder with
We run a shopify store. We integrated builder via the content api. We’re using the shopify Checkout Extensibility. We set up the tracking according to this example: Shopify custom pixel for builder to work with checkout extensibility - #5 by TimG

I’m very interested in other experiences and comparisons of other tracking tools vs. builder. Builder.ios support has been not helpful so far and is taking forever so I thought I put out this post to warn everyone about this potential bug. Please double-check your tracking with other tools and do not rely on builder.io as a single source of truth when it comes to conversion-tracking with shopify!

1 Like
2

Paul,

Thanks for sharing your analysis on tracking results, the discrepancy is being discussed internally, expect further details from Builder shortly.
Thanks.

3

Looks like we found a solid solution to the problem. The problem is the pixel code suggested here:

We threw the Code at Claude and got this one back which tracks 99% of all conversions:

const BUILDER_STORAGE_KEY = 'builderPendingEvents';
const API_KEY = 'Your API Key';
const MAX_RETRIES = 3;
const RETRY_DELAY = 1000;

// Helper function to safely get storage data
const getPendingEvents = () => {
    try {
        const stored = localStorage.getItem(BUILDER_STORAGE_KEY);
        return stored ? JSON.parse(stored) : [];
    } catch (e) {
        console.error('Error reading pending events:', e);
        return [];
    }
};

// Helper function to safely store data
const storePendingEvent = (eventData) => {
    try {
        const events = getPendingEvents();
        events.push(eventData);
        localStorage.setItem(BUILDER_STORAGE_KEY, JSON.stringify(events));
    } catch (e) {
        console.error('Error storing pending event:', e);
    }
};

// Helper function to remove stored events
const removePendingEvents = () => {
    try {
        localStorage.removeItem(BUILDER_STORAGE_KEY);
    } catch (e) {
        console.error('Error removing pending events:', e);
    }
};

// Function to send tracking data to Builder.io
const sendToBuilder = async (eventData, retryCount = 0) => {
    try {
        const response = await fetch(`https://cdn.builder.io/api/v1/track?apiKey=${API_KEY}`, {
            method: 'POST',
            body: JSON.stringify({ events: [eventData] }),
            headers: {
                'content-type': 'application/json',
            },
            mode: 'cors',
            keepalive: true,
        });

        if (!response.ok) {
            throw new Error(`HTTP error! status: ${response.status}`);
        }

        return true;
    } catch (error) {
        console.error('Tracking error:', error);
        
        if (retryCount < MAX_RETRIES) {
            await new Promise(resolve => setTimeout(resolve, RETRY_DELAY * (retryCount + 1)));
            return sendToBuilder(eventData, retryCount + 1);
        }
        
        // If all retries failed, store the event for later
        storePendingEvent(eventData);
        return false;
    }
};

// Main tracking function
const trackConversion = ({ checkout, amountInfo, meta = {} }) => {
    if (!checkout || !checkout.order || !checkout.order.id) {
        console.error('Invalid checkout data received');
        return;
    }

    const sessionId = document.cookie.split('; ')
        .find(row => row.startsWith('builderSessionId='))
        ?.split('=')[1] || null;
    
    const visitorId = localStorage.getItem('builderVisitorId') || null;

    if (!sessionId || !visitorId) {
        console.warn('Missing sessionId or visitorId');
    }

    const eventData = {
        type: 'conversion',
        data: {
            amount: amountInfo.amount,
            metadata: {
                sdkVersion: '3.2.10',
                url: location.href,
                orderId: checkout.order.id,
                currency: checkout.currencyCode,
                timestamp: new Date().toISOString(),
                ...meta,
            },
            ownerId: API_KEY,
            userAttributes: {
                sessionId,
                visitorId,
            },
            sessionId,
            visitorId,
        },
    };

    return sendToBuilder(eventData);
};

// Setup main event listener
analytics.subscribe('checkout_completed', (event) => {
    trackConversion({
        checkout: event.data.checkout,
        amountInfo: event.data.checkout.totalPrice,
        meta: { 
            additionalData: 'Conversion Recorded',
            eventId: event.id,
            clientId: event.clientId
        },
    });
});

// Try to send pending events on page load
window.addEventListener('load', async () => {
    const pendingEvents = getPendingEvents();
    if (pendingEvents.length > 0) {
        const results = await Promise.all(
            pendingEvents.map(event => sendToBuilder(event))
        );
        
        if (results.every(Boolean)) {
            removePendingEvents();
        }
    }
});

// Backup tracking before page unload
window.addEventListener('beforeunload', () => {
    const pendingEvents = getPendingEvents();
    if (pendingEvents.length > 0) {
        // Use synchronous localStorage to ensure data is saved
        try {
            navigator.sendBeacon(
                `https://cdn.builder.io/api/v1/track?apiKey=${API_KEY}`,
                JSON.stringify({ events: pendingEvents })
            );
            removePendingEvents();
        } catch (e) {
            console.error('Error sending beacon:', e);
        }
    }
});
4

Hello @pandus1s,

We are really glad to hear that. Feel free to reach out if you have any further questions or concerns.

Thanks,