Alessandro Desantis

In our [latest article](/blog/ecommerce-ab-testing), we wrote about the value of A/B testing (also
referred to as “split testing”) and how eCommerce brands can set up and scale an effective A/B testing strategy. By
getting the fundamentals right, brands can use A/B testing to generate unbiased, data-informed market insights. Large
brands often run tens of experiments at any given time and use the results to directly inform their product strategy.

However, we also touched briefly on the hidden complexity behind an A/B testing operation: in fact, **one of the curses
of A/B testing is its apparent simplicity.** It’s easy to get lured into signing up for an A/B testing tool without a
clear understanding of how it works, only to find ourselves weeks later with tests that are inconclusive or just plain
incorrect.

This article will dig deeper into this complexity, highlighting some of the most common pitfalls brands face when
running experiments. We will also discuss the scenarios in which A/B testing is _not_ the way to go and the alternative
practices brands should adopt instead.

## The 7 Most Common A/B Testing Mistakes

A/B testing is a form of statical hypothesis testing. As such, it is prone to all the typical pitfalls of statistical
analysis: most notably, **if the data you’re using to run the analysis is incorrect, biased, or skewed in any way, the
results of your analysis are also very likely to display the same problems**–garbage in, garbage out.

Even with modern A/B testing platforms, it is still unbelievably easy to skew the results of an A/B test. Your tools can
help you write less code and run the math for you, but they won’t be able to make strategic decisions for you–such as
which metric to pick–or to shield your test from external factors–such as a concurrent A/B test skewing your test
population.

With this in mind, let’s see some of the most common mistakes and oversights a brand might make when designing,
executing, and analyzing an A/B test.

### 1. Type I and Type II errors

All A/B tests are subject to two different types of errors:

* **Type I errors (false positives)** occur when we detect a non-existent difference between the experiment and the
  control group, i.e., we conclude that our experiment affected our target metric when it actually didn’t.
* **Type II errors (false negatives)** occur when we fail to detect a difference between the experiment and the control
  group, i.e., we conclude that our experiment failed to affect our target metric when it actually did.

We can guard our tests against Type I and Type II errors by picking an appropriate statistical significance and power:
the _higher_ our significance, the more resilient our test will be against Type I errors; the _lower_ our power, the
more resilient our test will be against Type II errors. Combined, these numbers represent our desired confidence in the
test result.

As an example, if we pick a significance of 80% and a power of 5%, it means that:

* There’s only a 20% probability that our experiment group will outperform our control group by chance.
* There’s only a 5% probability that our control group will outperform our experiment group by chance.

**Higher confidence levels mean we can take more drastic actions as a result of our test but also require larger test
populations,** which may be challenging to reach for smaller brands.

An 80% significance and a 5% power is a good starting point for most A/B tests, but you should use your judgment: in
some tests, the cost of making a mistake in either direction might be higher or lower than usual; in others, you might
only want to worry about false positives or false negatives.

### 2. Novelty Effect and Change Aversion

Two interesting dynamics can threaten the validity of A/B tests, and they both stem from how your users react to change.
These are more relevant for products with a high percentage of returning users, so you should be extra careful if your
brand offers subscriptions, for instance, as they could prevent you from getting reliable results.

**The Novelty Effect occurs when users engage with a product more than usual because they’re fascinated by a new feature
or a change in an existing feature.** For instance, users might purchase more if you introduce a brand-new loyalty
program to drive incremental recurring revenue. However, as they adapt to the change, most people will gradually regress
to the average user behavior patterns.

Change Aversion is the exact opposite and often happens with significant redesigns of existing functionality: **you
might find that affectionate users engage less with your experiment,** as they were used to the previous way of doing
things and don’t like that their workflows got disrupted. Over time, they will likely get used to the new design and
re-engage with your brand as usual.

For both problems, **the solution is to segment your A/B test results by new and existing users.** By analyzing the
performance of these two buckets separately, you can more easily isolate the effect of your experiment from the effect
induced by novelty/change aversion. Of course, you'll have to figure out what counts as a “new” or “existing” user, and
it might differ depending on the test you’re running.

### 3. Network Effect

**The Network Effect occurs when users of an A/B test influence each other,** effectively “leaking” outside the bucket
they belong to (i.e., the control group interacts with the experiment, or the experiment group reverts to control). This
most often occurs with “social” functionality, where users interact with each other directly.

Consider the case of a second-hand marketplace that wants to offer users the ability to exchange goods directly without
a cash transaction. By definition, such a feature will require two users to participate: the sender and the receiver.
One quick solution would be to assign users randomly to the experiment or control group and only allow users in the
experiment group to use the new functionality. However, this might lead to a sub-par UX, causing frustration and skewing
the test results.

Instead, **you should cluster users into groups, with all users in the same group being more likely to interact with
each other.** For instance, you might cluster users by state, assuming that most of your marketplace’s transactions
happen within the same state. You would then assign each state to the control or experiment group. Users might still
interact across different states, influencing each other, but clustering helps minimize the likelihood of spillover.

The Network Effect is relatively uncommon in eCommerce, but it’s still worth understanding if your brand allows users to
interact with each other (as in the case of a marketplace). It can lead to unexpected results and can be incredibly
sneaky to detect. As a rule of thumb, you should always consider whether any tests you’re designing have the potential
to introduce a network effect and plan accordingly.

### 4. History Effect

**The History Effect occurs when an event in the outside world affects the results of your A/B test.** This can be
anything from a marketing campaign to a shopping holiday. These events are likely to change the behavioral patterns of
your visitors compared to the average, which will cause you to extrapolate incorrect insights.

For example, let’s assume you’re testing whether offering a discount in exchange for newsletter signups significantly
increases subscribers. If you decide to run the test during a site-wide sale (e.g., because of Black Friday/Cyber
Monday), your test might be inconclusive.

**To mitigate the History Effect, ensure you have solid processes to ensure you’re not running A/B tests concurrently
with major media coverage and PR events.** It also helps to run your A/B tests for at least two complete business
cycles, which typically translates into 2-4 weeks for most brands.

### 5. Simpson’s Paradox

**Simpson’s Paradox is a particularly sneaky type of error that occurs whenever you inadvertently introduce weighted
averages into an A/B test,** which can happen in a few cases:

* **When you adjust the traffic split for an A/B test during test execution.** Example: you start your test at 50%/50%,
  then change it to 90%/10% when you see that the experiment is outperforming the control.
* **When your website traffic segmentation is not consistent across different variants of your test.** Example: you
  segment your test by returning vs. new visitors, but you have 1,000 new customers visit the experiment, and only 500
  visit the control.

In these scenarios, you might find that any correlation identified in individual test segments disappears or is inverted
in your aggregate results. This is because the aggregate result calculation effectively becomes a weighted average, and
your larger segments will “overwhelm” the smaller ones.

You can find a more academic explanation of Simpson’s paradox
[here](https://learn.microsoft.com/en-us/archive/msdn-magazine/2005/december/test-run-software-testing-paradoxes), but
there are a few things you can do to prevent Simpson’s Paradox from finding its way into your test results:

* **Ensuring your test populations are accurately randomized** and evenly distributed among different test variants. In
  other words, you should have the same number of visitors from each segment to each variant of your test.
* **Running an A/A test** (i.e., a “fake” A/B test where both variants are the same) can be helpful to ensure proper
  randomization.
* **Looking at test results across different segments** ensuring and not just at the aggregates so that you can spot any
  inconsistencies.
* **Relying on your A/B platform’s built-in capabilities,** e.g., Optimizely’s
  [Stats Accelerator](https://support.optimizely.com/hc/en-us/articles/5326213705101-Stats-Accelerator-Acceleration-Under-Time-Varying-Signals),
  to make your tests more resilient to Simpson’s Paradox.

### 6. P-hacking

P-hacking is an extremely common A/B testing pitfall–so much so that it used to be _encouraged_ by A/B testing tools
such as Optimizely. Simply put, **P-hacking is the practice of changing a test’s original parameters to reach a
pre-determined conclusion.** This can come in different forms:

* Some teams stop a test as soon as it reaches statistical significance rather than letting it run for its established
  duration.
* Some teams increase the test population until they reach statistical significance.

**P-hacking is often caused by pressure from your leadership or digital marketing team** to get a specific result from an
A/B test or to maximize the impact of a successful experiment as quickly as possible. Unfortunately, this is not how
traditional A/B testing works: once you have established your sample size, you simply need to let your run test run its
course and evaluate the results only at the end.

Because this methodology isn’t particularly well-suited to the speed at which startups typically move, many A/B testing
platforms ended up implementing alternative algorithms. Optimizely, for instance, introduced Stats Engine in 2015, which
allows A/B testers to peek at test results without the risk of taking action prematurely–you can learn more about how it
works in
[this introductory article](https://www.optimizely.com/insights/blog/statistics-for-the-internet-age-the-story-behind-optimizelys-new-stats-engine/)
by the Optimizely team.

While features such as Optimizely’s Stats Engine or VWO’s Bayesian engine help A/B testing teams avoid pitfalls such as
P-hacking, they don’t eliminate the need for proper test planning.

### 7. Instrumentation Effect

While most pitfalls outlined in this article are statistical, the Instrumentation Effect is much simpler: **it occurs
when your analytics, A/B testing infrastructure, or test implementation don’t work correctly**, skewing test results.
Here are a few examples:

* Your analytics setup sometimes reports conversions twice, skewing the total number of conversions you use to evaluate
  your test results.
* Your A/B testing setup sometimes puts the same customer in different buckets throughout the same A/B test.
* One of your test variants has a user experience edge case that prevents specific customers from completing their
  interaction with the webpage.

Because these bugs happen at the very source of your data, no statistical methods can solve any of these problems for
you. Instead, you need to regularly and rigorously test every part of your A/B testing infrastructure:

* Before running an experiment, **make sure that all the metrics you are interested in are being reported accurately.**
* **Test your experiments for usability and integrity** with the same attention you’d reserve to a permanent feature.
* **Run A/A tests to verify that your A/B testing setup works correctly** and that your upstream data collection is
  reliable.

## A/B Testing Is Not a One-Size-Fits-All Methodology

Reading this article, you might think that we want to discourage eCommerce brands from running A/B testing, and you’d
be–at least partially–correct. It’s not that A/B testing doesn’t have its place in an eCommerce brand’s strategy. But
**the effort involved in planning, executing, and analyzing an A/B test, assuming that you want to get significant
results, is often not sustainable by most early-stage brands.**

Anyone not adequately trained in the statistical techniques behind A/B testing will have a tough time guarding their
tests against the pitfalls we’ve outlined–which, by the way, are a subset of all the different statistical and practical
errors A/B testers can incur. Tools can help mitigate some of these errors to an extent, but they can’t turn an
inexperienced team into data analysis experts overnight.

Furthermore, **A/B testing is very often not the best research methodology.** There are many scenarios in which A/B
testing falls short.

* First and foremost: **A/B testing, by its very nature, is focused on short-term gains.** As we have seen, these can
  sometimes come at the expense of long-term business results: focusing on click-through and conversion rates might come
  at the expense of retention and customer LTV.
* As a result of the above: **A/B testing tends to deprioritize disruptive innovation,** leading teams to focus on
  incremental gains instead—its typical application is in conversion-rate optimization. If your brand is working on a
  massive product launch, or a brand new membership program, A/B testing will not offer any meaningful insights into how
  your customers might respond.
* **The cost of designing and launching an A/B test is high:** every test you run effectively creates two “branches” in
  your digital experience. This means your team will have to tiptoe around your A/B test until it’s run its entire
  course, slowing down further innovation and improvements in that product area.
* **A/B testing is a quantitative and evaluative research methodology.** As such, it needs an initial hypothesis to
  validate–and one that can be clearly and unequivocally validated with a data-driven approach. Anything requiring
  generative or qualitative research, which often involves talking to your customers, is not a good fit for A/B testing.

## Start A/B Testing When You Have Product Management Fundamentals

So, are we suggesting that retailers shouldn’t A/B test? Not at all: A/B testing has its place, and when employed
correctly, it can be instrumental in improving a business’s KPIs and bottom line. To dismiss A/B testing as too
complicated to be worth the effort would be incredibly short-sighted and detrimental.

However, **many eCommerce businesses dive head-first into A/B testing without proper product management fundamentals.**
We’re talking about generative and evaluative research methodologies such as heatmaps, user interviews, user testing,
on-site surveys, session replays, historical analytics, feature flags, and many other techniques.

These practices have a broader set of potential use cases and are also a prerequisite for being able to A/B test
intentionally. Plus, they’re almost always _simpler_ to implement and leverage than an effective A/B testing
infrastructure!

The next time you–or someone else on your team–want to run an A/B test, stop for a moment and ask yourself: **is there a
more straightforward, more efficient way to answer this question/validate this testing hypothesis?** You might be
surprised about how many alternatives you have at your disposal.

In the next few weeks, we’ll be exploring precisely these product fundamentals, explaining when and how they are best
used, and how you can orchestrate them together to form the basis of a strong product management practice for your
eCommerce brand.

How to Make Your eCommerce A/B Tests Useless

eCommerce brands have a variety of techniques at their disposal for understanding their customers and optimizing their KPIs: most of these techniques have been borrowed from modern product management practices and can be directly applied to eCommerce–A/B testing is one of these.

A/B testing (sometimes referred to as “split testing”) is a technique that uses statistical algorithms to determine which variation of a given behavior is most likely to optimize for a predetermined quantitative outcome. **By using live traffic to analyze user behavior, A/B testing can provide an objective, data-driven answer to nebulous questions**, such as whether offering free shipping to your customers will significantly affect add-to-cart rates.

With that said, **A/B testing is one of many techniques brands can employ and arguably one of the most complicated to execute correctly**. Unfortunately, we often see brand founders and executives jump into A/B testing without clearly understanding the required skillset or effort, only to arrive at disappointing conclusions and abandon the practice altogether.

We firmly believe that A/B testing has its place. Still, we also have strong opinions about when it should and shouldn’t be employed and the requirements for running a successful A/B testing operation. In this guide, we will break down some of the complexity behind A/B testing, help you understand whether it’s a good option for your brand, and offer practical tips and considerations for rolling it out.

## A Primer on How A/B Testing Works

At its core, A/B testing is a simple concept: we design two variations of the same web page (version A and version B), distribute our traffic evenly among the two variations, then see which one reported the best performance.

Going back to the “free shipping” example, an online store could show a free shipping banner to half their customers (this is the experiment group) and use the original page for the other half (this is the control group). At the end of the test, whatever version got the highest conversion rate would be considered a winner (with some statistical caveats).

This is the simplest possible type of A/B testing, but there are a few alternatives:

* **In A/B/n testing**, more than two variations are tested (e.g., no free shipping indication vs. a free shipping banner vs. a free shipping popup).
* **In multivariate testing**, each variation alters multiple elements on the page (e.g., a free shipping banner and a popup, just the banner, just the popup, no free shipping indication).
* **In bandit testing**, more and more traffic is automatically directed to the winning variation (e.g., if the free shipping banner converts more than the original page, we automatically start showing it more and more often).

![MailChimp A/B testing](https://cdn.sanity.io/images/kqgmf0g2/production/9c659965ed7f4ff19fa622d3e58e7b19fc883353-2560x2512.png?fit=clip)

It’s important to note that while this article talks specifically about A/B testing web pages, the technique can be applied—at least to an extent—to email marketing, social media, and other types of digital properties. Furthermore, eCommerce platforms, marketplaces, and marketing tools often provide native A/B testing capabilities:

* Amazon allows its merchants to A/B test content such as product images
* Hubspot allows testing landing page templates and email campaigns.
* Google Ads allows testing different ads to optimize clickthrough rates.

These concepts are deceptively simple, which is why so many A/B testing platforms promise exceptional ROI and quick results. **In reality, A/B testing comes with many pitfalls and “gotchas” that practitioners should understand thoroughly**. Running an effective A/B testing operation on an eCommerce website requires significant preparation, effort, and discipline.

## What an Effective A/B Testing Process Looks Like

Let’s take a peek into what a successful, effective A/B testing operation looks like in detail.

### 1. Picking a Problem Space

There are many resources online advertising “the best 99 A/B testing ideas,” and it might be tempting for inexperienced executives to jump in and start testing. Unfortunately, these blanket experiments are unlikely to move the needle–in some cases, they might even be detrimental to a brand’s KPIs. Furthermore, they completely ignore the true purpose of A/B testing, which is not conversion rate optimization but refining our understanding of our market, business, and customers.

For this reason, before diving head-first into changing the color of our CTA buttons, we must understand the problem space we want to explore. This can be a very narrow product opportunity, such as whether a new version of the search functionality will lead to better recommendations, or a broader area of the business that we want to understand better, such as how we can improve our conversion rates during the checkout process.

![POA template](https://cdn.sanity.io/images/kqgmf0g2/production/a7dfcc5783e16119a81831284987627f9fcb093f-2560x2512.webp?fit=clip)

Whatever the case, **we need to articulate precisely what questions we want to answer and why we think an A/B test is the correct methodology to answer them**. This is most often done through a Product Opportunity Assessment (POA), a document that explains what problem we’re solving and for whom, analyzes the market and the problem space, any business- and customer-facing implications of solving the problem, and provides a final recommendation (go/no-go) based on the information we have collected.

A/B testing, when employed correctly, is an invaluable tool that allows eCommerce product managers to quantitatively confirm or disprove existing hypotheses related to a particular opportunity. Still, it’s not a replacement for picking and refining an opportunity in the first place. Unfortunately, brand executives and marketers sometimes fall into the trap of running mindlessly through a list of standard A/B tests, leading to short-term, black-box optimizations at best and a waste of resources.

### 2. Designing the Test

Once the opportunity is well understood and prioritized, it’s time to design the A/B test. The first step is formulating a hypothesis that we want to confirm or disproven, which usually looks like this:

> We think that, by [experiment], we can increase [metric] by an [absolute/relative] [Minimum Detectable Effect]%.

Stating the hypothesis is the first step, but we will need a few more elements before launching our experiment. In particular, we need to establish the following:

* **Our current conversion rate**. If we don’t have it, this is a good time to stop and put the proper instrumentation in place.
* **The desired statistical power**. This indicates how likely we are to detect a difference between the control and the experiment, assuming it exists. Higher values give us more confidence in the test results but require larger samples.
* **The desired significance level**. This indicates how likely we are to detect a difference between the control and the experiment when it doesn’t exist. Lower values give us more confidence in the test results but require larger samples.

![Evan Miller's A/B testing sample size calculator](https://cdn.sanity.io/images/kqgmf0g2/production/e69b939e8d2e7df81f552d122ff5d4ffd00fba9e-2560x1534.webp?fit=clip)

Once we have established these parameters, we can plug them into one of many A/B test sample size calculators–Evan Miller’s is arguably the most popular, but any popular A/B testing platform (e.g., **LaunchDarkly**, **Optimizely**, **VWO**) will also do the math for you.

Finally, we must decide how long we want to run our test. **It’s usually recommended to run tests for at least two complete business cycles**. This helps to protect our experiment from a few common pitfalls, such as external validity threats, novelty/maturation effects, and change aversion. We’ll cover these in a separate article!

### 3. Launching the Experiment

The next step is to work with our product team to design and implement our experiment.

Most modern A/B testing platforms allow non-technical users to launch tests with complete autonomy. After copy-pasting a snippet of JavaScript code to their website, brand operators can design and launch experiments that alter UI elements without touching a single line of code: users can alter page layout, CTA styles, and other elements. This capability allows product managers and stakeholders to move quickly and saves engineering bandwidth for high-impact product work. However, it also comes with some caveats of which users are typically unaware.

First of all, **experiments that have been implemented without the assistance of a product team might present bugs or inconsistencies that make them less palatable to new and existing brand customers**. This might skew our test results: what if our latest category page experiment failed to outperform the control because the customer experience was counterintuitive? In some cases, such pitfalls can completely invalidate an experiment, resulting in a waste of resources.

![Optimizely homepage](https://cdn.sanity.io/images/kqgmf0g2/production/f8e3234adf8087323de71b30402eb2e6bdc7133e-2560x1702.webp?fit=clip)

Secondly, it’s not uncommon for a large brand to run tens of experiments simultaneously. **When experiments run in the browser, they can have significant performance implications and harm the user experience**.

**Brands should invest in A/B testing infrastructure that allows them to design and launch experiments quickly while continuing to involve the product team.** This often comes in the form of streamlined processes, strong design/development primitives–such as a reusable design system and UI kit–and clear guidelines on what is considered “good enough” for an experiment.

### 4. Analyzing Test Results

Once the test is completed, it’s time to analyze the results. Typically, there are two questions we’re looking to answer for each A/B test.

**The first and most obvious is whether the experiment outperformed the control**–or, in other words, whether our hypothesis was confirmed or disproven by the test. If the experiment outperforms the control, we can say that our hypothesis is correct–within the established levels of statistical confidence, of course. We might also find that the control outperformed the experiment or that the test was inconclusive (i.e., neither variation significantly outperformed the other, and no winner can be established).

**The second question is what secondary implications the test had on our experiment population.** This might involve looking at other metrics or specific customer segments to understand whether the test affected our customers in non-immediate ways. Here are a couple of examples:

* A test might tell us that offering 50% off on the first order of our subscription product increases our subscription activation rate. However, when we look at secondary metrics, we might find that the experiment group also has a much lower retention rate than the control group, causing us to lose money on every conversion. Is this outcome desirable for our eCommerce business?

* A test might be inconclusive or present the control group as a global winner. However, when we dive into the data, we might find that the experiment blatantly outperformed the control for specific segments. Sometimes, this might indicate an opportunity to offer a more [personalized experience](https://nebulab.com/blog/ecommerce-personalization).

![LaunchDarkly metrics view](https://cdn.sanity.io/images/kqgmf0g2/production/35a9f4a0a28701605f61c311163f4345cd12eec8-1200x857.webp?fit=clip)

As a rule of thumb, it can be helpful to map some of these secondary considerations when designing the test. However, this analysis is very exploratory, and it’s impossible to list all the possible outcomes in advance. **Brands should take the time to review test results with different stakeholders so that everyone can bring their KPIs and perspective to the table.**

Once we have finished analyzing the results, it’s time to archive them. This is a step that most practitioners skip, but it’s incredibly valuable: **archiving the results of our A/B tests allows us to establish them as a first-class research methodology** rather than a short-term approach to conversion optimization. Ideally, insights derived from A/B tests should systematically become part of our organizational knowledge.

### 5. (Bonus) Re-Testing as Needed

What’s true today might not be true tomorrow. When they first embrace a product-led approach to brand growth, we see many founders make the mistake of thinking that any insights they have uncovered are set in stone and will never need to be revisited. In reality, markets change constantly, and so does our business.

We don’t encourage re-running A/B tests for the simple sake of it–in most situations, the effort of re-testing far outweighs the benefits. However, **practitioners should feel comfortable re-running an old test or testing old assumptions in novel ways**.

## An End-to-End A/B Test Example

Let’s see what an end-to-end A/B test execution might look like by following the process we have just outlined:

1. As we’ve mentioned, it all starts with identifying a problem space we want to explore. In this case, let’s assume we are trying to understand why our product page has a particularly high bounce rate. We draft a Product Opportunity Assessment, collecting all the information we have gathered.

2. We collect a few hypotheses about why this might be happening, based on what we know about the business and our customers: maybe visitors can’t find critical information in the product description; maybe the particular variant they are looking for is out of stock; or maybe they aren’t willing to pay shipping costs.

3. We start by testing the shipping cost hypothesis. After looking at Google Analytics, we conclude that increasing add-to-cart rates by an absolute 10% would be considered a win.

4. Because offering free shipping comes at a considerable cost, we want to be very confident about our test results, so we pick a 95% statistical power and a 10% significance level for our test.

5. We plug the numbers into our favorite A/B test sample size calculator, which tells us we’ll need to direct 858 visitors to each branch of our test. Based on our traffic patterns, we think we can accomplish this in two weeks.

6. We work with our design and engineering team to implement an experiment that offers customers free shipping and displays the offer prominently on the product page.

7. We use our favorite A/B testing tool to launch the test.

8. After two weeks, we look at the results of the test. It turns out that the test was overall inconclusive: offering free shipping did not significantly affect the add-to-cart rate. However, when we look into the results across different segments, we find that the experiment group outperformed the control group for products over $150.

9. We archive the results of the test and refine our POA accordingly.

![LaunchDarkly results view](https://cdn.sanity.io/images/kqgmf0g2/production/bb3782925740b9b70a5349429382e14e496c3113-1200x860.png?fit=clip)

Now that we have the results of the test, we may decide to:

* Offer free shipping to everyone. Our business might have a policy of offering the same shipping conditions to everyone.

* Offer free shipping only on products over $150 to maximize our ROI. We might even play with different placements of the free shipping indication: what if we included the banner in the shopping cart or checkout page?

* Run another A/B test on a more promising target audience. We could test free shipping for products over $150, to be confident that it will actually move the needle for that segment.

* Collect more information through alternative product research methodologies like user testing or heatmaps.

* Not do anything: a more detailed evaluation might find that free shipping is not the highest-impact solution to this problem (e.g., perhaps we’d be better off adding testimonials to the product page or offering an upsell).

As we’ve mentioned, **an A/B test doesn’t have to lead to immediate action**. By treating our tests as a source of insights rather than a mere CRO methodology, we can deepen our knowledge of our customers with statistically verifiable, data-driven information that will enrich our product strategy and ultimately increase our bottom line.

## What Comes Next

We could talk for ages about the pros, cons, prerequisites, and implications of A/B testing, but this is more than enough for a single article.

**In the following articles, we’ll explore the most common pitfalls operators, marketers, and product managers incur when setting up and executing an A/B testing strategy**. We’ll talk about statistical significance, the shortcomings of using A/B testing as your only source of product knowledge, and what alternative strategies brands must employ before considering running an A/B testing operation.

In the meantime, if you have any questions about any topics we have discussed or need help designing an A/B testing strategy for your eCommerce site, [don’t hesitate to reach out!](https://nebulab.com/contact-us)

eCommerce A/B Testing: Strategies for Success

Daniel Bergholz

While the total penetration of headless commerce in the US is still low, we have witnessed an increase in headless builds in recent years. And even though [we don’t believe headless is the end-all solution to all eCommerce problems](https://nebulab.com/blog/headless-commerce), we think it’s about time the industry realized the power of this architecture and started using it responsibly.

The only way for Shopify users to go headless for a long time was to implement their own single-page application from scratch, using a JavaScript library such as React (and hopefully a framework, such as Next.js). Shopify does provide SDKs to make the API integration work slightly more manageable, but that was about it: until recently, there was very little being done for performance, state management, personalization, or SEO.

In November 2021, Shopify released a developer preview of [Hydrogen](https://hydrogen.shopify.dev/), their new framework meant to address precisely these problems. Here at Nebulab, we’ve recently had the chance to take Hydrogen for a spin, and we want to share our learnings.

## What is Hydrogen?

Hydrogen is an open-source, React-based framework that can be used to build custom storefronts on the Shopify platform. It provides a lot of useful functionality:

* A sensible structure for your application code.

* eCommerce-specific components and pages.

* An SDK and hooks to interact with Shopify’s GraphQL-based [Storefront API](https://shopify.dev/api/storefront).

* It uses [Tailwind CSS](https://tailwindcss.com/) for styling.

* It’s optimized for high-performance builds.

Combined, these features almost entirely eliminate the amount of boilerplate code engineers need to write to build their storefronts and allow engineers to spend most of their time on functionality that truly moves the needle.

![Shopify Hydrogen website](https://cdn.sanity.io/images/kqgmf0g2/production/9865cb4da04e7cd861e38c97cc7015d8ea7a1e22-2560x2512.png?fit=clip)

## What are the pros of using Hydrogen?

For Shopify stores that want to go headless, Hydrogen is a huge help. As we’ve mentioned, it abstracts away all the complexity of integrating with the Shopify API and developing your storefront’s skeleton: all you need to do is clone the Hydrogen [demo store](https://shopify.dev/custom-storefronts/hydrogen/getting-started/templates#demo-store-template), configure it, then focus on styling and custom functionality. This alone is a huge productivity boost, and the developer experience has been outstanding in our experiments.

It’s also important to note that Hydrogen is officially developed and supported by Shopify, which should provide peace of mind to brands looking for a stable, robust solution that’s not going anywhere. While some Shopify-specific Next.js templates already existed before Hydrogen, they were mostly small, community-sponsored projects, which made them unsuitable for enterprise use.

## What are the cons of using Hydrogen?

Seasoned developers know that productivity comes with a hidden cost: the more a tool tries to abstract away technical complexity, the more opinionated it is. When it comes to Hydrogen, it’s exceptionally opinionated:

* It uses React server components.

* It uses streaming server-side rendering (SSR).

* It has built-in components, hooks, and caching mechanisms.

* It’s heavily coupled with Shopify’s Storefront API.

* If you need complete control over your React application, Hydrogen is most likely not a good candidate for you.

You should also know that some of these features are bleeding edge in the React ecosystem, and because Hydrogen is tightly coupled with Shopify, the community around it is still relatively small. Hydrogen brands can expect to have a harder time finding online resources and support, and they will have to tap into a smaller pool of agencies and engineers than if they used a more popular framework. That said, much of that will likely change as Hydrogen gains popularity.

Finally, most Shopify Apps and plugins were built to integrate with Liquid templates and won’t be compatible with Hydrogen (or other headless architectures, for that matter). If you rely heavily on specific apps and your app doesn’t provide an API your engineers can integrate with, you’ll need to switch to a headless-friendly alternative or build the functionality yourself.

## Can Hydrogen be used without Shopify Plus?

It’s complicated. The official answer is yes: since Hydrogen is an open-source project whose source code is available to anyone, and because none of the API endpoints it uses are restricted to Shopify Plus customers, at first glance, the framework can be used without Shopify Plus.

In reality, things are more nuanced.

![Shopify Oxygen documentation](https://cdn.sanity.io/images/kqgmf0g2/production/a893aabf4ed6f21f992598100837e4856353c79f-2560x1620.png?fit=clip)

First of all, there’s the matter of hosting your Hydrogen application. Plus users can host their Hydrogen app on [Shopify Oxygen](https://shopify.dev/custom-storefronts/oxygenhttps://shopify.dev/custom-storefronts/oxygen), Shopify’s proprietary hosting solution, which provides native support for directing users who visit your Shopify domain to your Hydrogen storefront.

If you don’t use Shopify Plus, you should still be able to deploy your Hydrogen app to other platforms, but it won’t be a painless experience. For example, Vercel has published a blog post announcing the availability of a [zero-configuration deployment for Hydrogen](https://vercel.com/changelog/hydrogen-projects-can-now-be-deployed-with-zero-configuration). But when we tested it, we had to manually tweak the environment variables used by Hydrogen to match the names used by Oxygen. This was a minor inconvenience overall, but it shows how support for Hydrogen is still in its infancy.

Once you’ve figured out the hosting, it’s time to deal with domains and DNS records. If you manage your domains through Shopify, Shopify will attempt to redirect any requests to your [myshopify.com](http://myshopify.com/) domain to your Hydrogen storefront, which will completely break the checkout. This could be easily solved by using a [routing domain](https://help.shopify.com/en/manual/domains/domain-type/routing-domain) which, ironically, is only available to Shopify Plus customers. (When we reached out to Shopify engineering about this catch-22-esque paradox, they seemed utterly taken aback by this realization.) The current solution is to set up your Liquid storefront to redirect to your Hydrogen storefront via JavaScript, which is far from ideal.

Another aspect to consider is support: in our experience, if you’re not a Shopify Plus customer, you’re only going to get extremely limited technical support from Shopify, with most requests being redirected to Shopify’s Discord community. No one in the Discord community seems to have much experience with Hydrogen, except for Shopify’s own employees, whose presence in the community is very erratic.

To be fair, Shopify did tell us that they are working to address these challenges and make Hydrogen truly available to non-Plus customers. However, _when_ this will happen exactly is unknown.

To sum it up, if you’re not on Shopify Plus and don’t have an experienced engineering team or agency backing you up, we would advise against using Hydrogen right now: you run the risk of spending weeks or months of development only to run into deal-breaking roadblocks.

![Shopify Discord](https://cdn.sanity.io/images/kqgmf0g2/production/ce5f2c4f4565702286b8a1df6cd0d17d418c8538-2560x2068.png?w=450)

## Who should be using Hydrogen?

Shopify’s Hydrogen framework is the perfect solution for online stores that are already running headless, or are planning to go headless, can count on a robust engineering team, and want to lean into a powerful, officially-supported solution that will spare them from boilerplate work. The developer experience is enjoyable, and most React engineers will be more than happy to work on a Hydrogen app.

If, on the other hand, you don’t have an experienced engineering team or don’t want to deal with the complexity that comes with a headless solution, Hydrogen is not a good choice: as much as it simplifies things, it will still require much more upkeep than a Liquid-based Shopify build.

At this point, Hydrogen is also a less-than-ideal solution for brands not on Shopify Plus due to missing functionality and limited support from Shopify. That is likely to change as the framework matures and becomes more widely available, so keep an eye out for any announcements from Shopify (and from us!).

If you’re considering going headless with Hydrogen or other frameworks, and are unsure of the implications, [feel free to reach out](https://nebulab.com/contact-us)! Our team has extensive experience and will help you make the choice that best supports your digital strategy.

Everything You Need to Know About Shopify Hydrogen

With unlimited digital estate, DTC brands can tell their stories, announce, entertain, educate, and keep their name front and center in customers’ minds, all of which is very hard to do in the limited shelf space afforded by wholesale partners. Now that even die-hard digitally-native brands are going omnichannel in pursuit of reach and profitability, **it’s paramount to double down on the exclusive capabilities of each channel to create complementary experiences—for DTC, content is one of the most obvious levers to pull**.

A slew of platforms has emerged over the past decade that promise to offer the definitive solution to all content management pains. These solutions provide functionality that covers the entire content management lifecycle, from writing to publishing, presentation, and SEO, but they do it in very different ways. In fact, we’ve seen at least four generations of content management tooling emerge in these years, all of which coexist today, serving different use cases and market segments.

For eCommerce operators, finding a North star to follow in this vastness can be tricky, and engineering teams are usually not much better off. If you haven’t been exposed to all these tools and understand the nuances of each, it’s easy to get lured in by the promise of unlimited flexibility and endless automation, only to find yourself stuck with a complex piece of software that requires weeks of integration work, or doesn’t play nice with your editorial workflow.

In this article, we will provide an overview of the content management space, as seen from the perspective of an eCommerce brand. You will learn the different types of tools you can use, their advantages and limitations, and some practical recommendations about which tool to adopt depending on your goal, use case, and growth stage. The goal is to equip you with all the knowledge you need to make an informed decision for your eCommerce brand.

## The four generations of content management

Different content management solutions have been launched in the last twenty years. While there’s no universal taxonomy, we can classify them under four categories: monolithic CMSs, headless CMSs, site builders, and page builders. In the following paragraphs, we’ll explore these different solutions, their pros and cons, and the leading players in each space.

### Monolithic CMSs

Monolithic CMSs were the first content management solution to emerge in the market. **These tools take over your entire content experience by handling every single aspect**: they provide an interface that allows you to edit and publish your content and present the content to your audience via a public-facing website.

You’ve probably heard about [WordPress](https://wordpress.org/), arguably the most popular CMS in the world. Initially built for blogging, WordPress has been expanded over the years to encompass an incredible number of use cases, allowing you to publish blogs, entire websites, and even online stores. It’s estimated that WordPress powers over 43% of all websites.

![WordPress website](https://cdn.sanity.io/images/kqgmf0g2/production/908b2d57d03ec193412387c8af8d151b6df2393b-2560x2136.png?fit=clip)

**Monolithic CMSs have been around for over fifteen years, and they’ve stood the test of time**. They offer a user-friendly, streamlined content editing experience and many plugins and apps that make them easy to repurpose for different use cases. Since they have a considerable market share, they also boast rich ecosystems of system integrators.

However, they also come with some limitations: th**ese content management systems are meant for content-centric websites, not for eCommerce brands.** They usually take over your entire website and don’t play nice with other parts of your tech stack. This makes them a less-than-ideal choice for a DTC brand, which needs to present the content on their storefront or even enrich it with dynamic data such as product information.

### Headless CMSs

Headless CMSs were built to solve precisely these limitations. **Compared to monolithic solutions, headless CMSs are much more nimble**: a headless CMS only provides an interface to define your content structure and manage your content. Presenting your content is up to you, and typically requires a design and engineering team to integrate the headless CMS into your existing frontend.

[Contentful](https://contentful.com/) has long been the primary contender in the headless CMS space, but many competitors have launched their solutions over the years: [Prismic](https://prismic.app/), [DatoCMS](https://datocms.com/), and [Sanity](https://sanity.io/) are among the most popular alternatives, offering slightly different functionality such as custom editorial workflows/UIs and internationalization, and wildly different price points.

![Contentful homepage](https://cdn.sanity.io/images/kqgmf0g2/production/551c31bc9b5a60be3f663151ba0172e06f790da5-2560x1810.png?fit=fill)

**Headless CMSs enable brands to treat content as data, making it fully portable and allowing them to present their content wherever they want**. Most headless CMSs also connect to your backend: for instance, an eCommerce brand will be able to pull product and pricing information from Shopify and include it right in the middle of an editorial piece. Such functionality allows you to blur the lines between content and commerce, making for a much richer customer experience.

Headless CMSs are still very popular thanks to their flexibility, but they come with their own limitations: mainly, **they require design and engineering work when marketers want to integrate new types of content into a company’s existing digital properties, or whenever they want to make visual changes to the existing experience**. This may be problematic for brands operating with low budgets or in high-velocity environments.

### Site builders

While site builders are not technically new (Webflow was born in 2012, one year before Contentful), they’ve seen a recent surge in popularity. You can think of site builders as a modern take on monolithic CMSs: like monolithic solutions, site builders also take over your entire website. However, **they offer more modern and advanced functionality**, such as support for drag-and-drop editing across your entire website, responsive design, custom code, API access, and integrations with popular third-party design and development tools. Almost all site builders also include a native headless CMS, or will optionally integrate with a third-party headless CMS.

The most popular site builder is [Webflow](https://webflow.com/), which is often used by SaaS companies to manage their marketing websites, or by eCommerce brands to spin up self-contained editorial experiences. Webflow also allows creating gated content and building Zapier-esque workflow automation. A recent competitor is [Framer](https://framer.com/), which allows designers to build, test and publish websites straight from Figma wireframes.

![Webflow homepage](https://cdn.sanity.io/images/kqgmf0g2/production/0082e34207f075b05269676c871a8d0d49b1601a-2560x2034.png?fit=fill)

**If your team has limited technical experience or you don’t want to use your engineering bandwidth, using a site builder can be an excellent way to quickly spin up new editorial experiences that will meet the standards of the modern Web.** And if you need to implement deeper customizations or complicated workflows, your engineers can still make those adjustments at a reasonable pace.

The main disadvantage with site builders is that, for all their flexibility, **they will still be pretty self-contained, and they’re hard to integrate with other digital properties.** Maintaining brand consistency will also be more challenging since your design system will live both in your site builder and across any other frontends you have. This is fine if you’re building a standalone experience such as a microsite, but can hurt your brand reputation when used for integrated experiences such as a company blog.

### Page builders

After coming full circle with site builders, the content management pendulum is swinging the other way once again with page builders. Like site builders, page builders also offer drag-and-drop editing, built-in headless CMSs, and all the modern developer tooling you might expect. **Unlike page builders, site builders try to be non-obtrusive: rather than taking over your entire website, they allow you to model specific pages or sections of it.** For example, you might let your eCommerce platform power your product catalog and checkout experience while your page builder powers your site navigation and blog.

The most popular page builder is [Builder.io](http://builder.io/), used by several eCommerce brands such as Everlane, J.Crew, and Harry’s. It allows you to use their native visual editor or to import your own components, connects with several third-party data sources, offers built-in A/B testing, and lets you publish your content across the Web, mobile, and wherever else you want. An early-stage contender is [Plasmic](https://plasmic.app/), which offers a very similar feature set, but pairs it with a more modern UX.

![Builder.io homepage](https://cdn.sanity.io/images/kqgmf0g2/production/29e681ed0cb541c8da8d8c65119b10cfe3f08b8d-2560x1528.png?fit=clip)

**Page builders can be an excellent compromise between headless CMSs and site builders**: they allow you to easily integrate your content into your existing digital properties in a non-obtrusive, brand-consistent way, and they offer a visual interface that helps non-technical stakeholders to quickly iterate on both content and presentation.

However, they are not a one-size-fits-all solution. **While an essential page builder integration can be a reasonably low-effort process for your engineering team, getting page builders right can be a complex initiative.** Once you start bringing in your own UI components, performing search engine optimization, running A/B tests, integrating with external data sources, or implementing rich user interactions, the level of complexity goes up dramatically, and keeping everything running smoothly can be a challenge.

## Evaluating the right solution for your brand

As you’ve seen, there’s no easy answer: each eCommerce CMS has its pros and cons, which you should weigh against your company’s stage, requirements, and technical savviness. The three main factors we keep into account when picking a solution for our clients are:

* **Business goals**: what do you want to achieve with your content? This is closely tied to where you’ll want to place your content, which in turn informs your choice of tooling. Are you looking to inform, inspire, or convert? (Often, it will be a mix of these, which is perfectly fine—just make sure you cover all your bases!)

* **Span of control**: site builders give operators maximum flexibility to change whatever they want, whenever they want it. This is both a blessing and a curse: a non-designer making design changes can quickly become a disaster. What’s the sweet spot between not enough and too much control for your team?

* **Implementation effort**: page builders are more complex to integrate than headless CMSs, which in turn are more complex to integrate than a monolithic CMS. Do the benefits outweigh the extra effort of integrating the solution with your tech stack?
Apparel brand and Nebulab client [Huckberry](https://huckberry.com/), for instance, relies heavily on their blog, [The Journal](https://huckberry.com/journal). The Journal is a fully integrated part of Huckberry’s shopping experiences: articles typically feature one or more Huckberry products, which users can add to their cart with a simple click. It’s a very conversion-centric approach.

![Huckberry’s blog, The Journal](https://cdn.sanity.io/images/kqgmf0g2/production/7ce46f90390f88961bbc568a2fafc78deb24b464-2560x2512.png?fit=clip)

Travel brand [Away](https://awaytravel.com/), on the other hand, publishes the travel and lifestyle magazine [Here](https://www.heremagazine.com/). Away’s magazine has a laid-back, inspirational approach with its content, and you can hardly notice that a DTC brand is behind the project.

![Away’s magazine, Here](https://cdn.sanity.io/images/kqgmf0g2/production/85383a0a4b600d7279c82359e0f0da9edf9209b8-2560x1664.png?fit=clip)

### Picking tools vs. building infrastructures

**Rather than only adopting one solution, we recommend creating a content management infrastructure, i.e., a suite of tools that integrate and build on top of each other.** For example, here’s a typical journey a DTC brand might undergo:

1. As a first step, they adopt a headless CMS, which allows them to decouple content and presentation. They use their headless CMS as the source of truth for all their content and integrate it with their existing digital properties (e.g., their product pages).

2. At some point, the marketing team starts complaining that they can’t make visual changes to how the content is presented on their product pages without asking the engineering team. This leads the team to integrate a page builder for specific website sections (e.g., the PDP and the blog). The page builder is connected to their headless CMS, which is still used for managing the actual content.

3. The brand eventually outgrows the page builder as well: the marketing team wants to start experimenting with standalone experiences such as microsites for contests, promotions, and product launches. This is when the team adopts a site builder, giving marketing full power to launch new properties with little to no engineering assistance.

**As marketing teams gain more freedom, brands need to create checks and balances to avoid brand dilution.** This can mean having a content inventory that is continuously monitored for consistency, ensuring the same person is orchestrating all digital experiences, or having a full-fledged design system that can be applied across different platforms. **Ultimately, how much centralization you need depends on your brand’s scale and your reliance on content.**

## Our favorite CMSs for eCommerce brands

At Nebulab, we have helped tens of brands design, launch and optimize their content stack, and we have developed some pretty strong opinions on which CMS platforms to use:

* **Monolithic CMS:** [WordPress](https://wordpress.org/). As mentioned, we rarely pick monolithic CMSs for eCommerce brands—but when we do, we go with WordPress. It has a beautiful, intuitive UX and an incredibly vast ecosystem of plugins and integrations, making it a good fit for many use cases and deployment options.

* **Headless CMS:** [Sanity](https://sanity.io/). A few years ago, we would have recommended Contentful or Prismic ([and we have](https://nebulab.com/blog/five-no-code-ecommerce-management-tools#the-best-headless-cms-prismic)). Ever since we’ve had a chance to try Sanity, we fell in love with its flexibility. While traditional headless CMSs force users to use a standardized UI for managing content, Sanity’s content studio is open-source and can be tailored to your content management needs and editorial workflows.

* **Page builder:** [Plasmic](https://plasmic.app/). While Builder.io has been around for slightly longer, Plasmic offers a more modern UI for their editing backend. With that said, the two tools are incredibly similar, so we encourage you to experiment with both and see which one you prefer.

* **Site builder:** [Webflow](https://webflow.com/). Even though products like Framer offer a more streamlined experience for non-technical users, Webflow remains an incredibly powerful tool if you know how to use it. With support for custom workflows and gated content, your team can quickly spin up rich micro-sites that cover various use cases.

As always, your mileage may vary. Before committing to a solution, you should thoroughly assess your present and anticipated requirements and experiment with different setups to build a stack that can scale with you as your needs evolve. If you need any advice or want to speed up the process, [we’re happy to help](https://nebulab.com/contact-us)!

Building Content Infrastructures for eCommerce Brands

With 9 out of 10 online shoppers stating that they want retailers to offer recommendations based on their interests, and
considering the vast amount of data and technology brands now have, there are few excuses not to have a personalization
strategy in place. eCommerce personalization is proven to increase conversion rates, customer retention, and lifetime
value, creating a virtuous cycle that can turn your visitors into loyal customers.

And yet, we often see brands struggle with rolling out an effective eCommerce personalization strategy: founders aren’t
sure how to segment their customers, evaluate the effectiveness of their personalization program, or what technologies
to adopt.

This article will demystify eCommerce personalization, providing you with a straightforward, data-informed approach for
designing, implementing, and validating your personalization program.

## What Is eCommerce Personalization?

Much confusion stems from the vagueness of the term “personalization.” If you Google it, you’ll find books, articles,
and SaaS products that all give it a different meaning: they talk about A/B testing, machine learning algorithms,
retargeting, user-generated content… It can be hard to understand where to start.

At its core, eCommerce personalization is the technique of using the information you have about your customers to
display content that is tailored to them. You can personalize across different dimensions (e.g., personalize by country,
browsing history, demographics), at different touchpoints (e.g., your website, your email campaigns, your ads), and with
different techniques (e.g., machine learning, user-generated content, human curation). No matter *how* you personalize,
the goal is to create a customer experience that feels unique.

## Designing Experiments with GTDT

When we coach founders that want to start experimenting with personalized eCommerce, we encourage them to think of
personalization as the means, not the end. Like all initiatives, personalized shopping experiences must be designed
thoughtfully and intentionally for maximum effectiveness. Adopting a tool or installing a plugin will only take you so
far if you don’t have a clear understanding of what you want to achieve with your personalization program.

Unfortunately, most resources offer an endless list of personalization examples from famous online retailers but no
indication of *why* or *how* those brands designed their personalization programs. This leaves operators scratching
their heads as they try to replicate the success of these brands. As a result, they tend to go for “blanket solutions”
that are easy to implement but ultimately leave the most exciting opportunities on the table.

Over time, we have developed a model that helps brands roll out thoughtful personalization experiments that get them
closer to their goals. **We call it GTDT, which stands for Goal, Technique, Dimension, and Touchpoint.** In the
following paragraphs, we will explain this process, providing you with all the tools you need to design your
personalization program.

### Goal

This might seem obvious, but surprisingly often, online stores will launch personalization capabilities to tick off a
checkbox with no clear goal in mind. Unfortunately, this not only means they will roll out an ineffective
personalization program, but it also means they will not be able to validate the effectiveness of their program: if you
don’t know what metric you’re optimizing for, you will not know what metric to look at.

The first step for any good personalization experiment is to understand what goal you’re aiming for—everything else
follows. Depending on your brand’s SKU catalog and business model, a personalization experiment for increasing your
average order value will likely look much different from an experiment to improve your add-to-cart or repeat purchase
rates.

If you’re unsure what goals you should be prioritizing, talk to your team and pick the one that feels most urgent right
now. Where are you falling behind compared to the industry benchmarks and your (or your board’s) expectations? What are
your customers telling you in your interviews? (You ***are*** doing customer interviews, right?) What are your plans for
the next 12-18 months?

Don’t make the mistake of falling into analysis paralysis: the beauty of running experiments is that you can quickly
pivot if you find that you’re optimizing for the wrong things. Make a data-informed decision, launch your experiment and
move on!

### Technique

With eCommerce personalization rising in popularity, several tools have emerged that promise to offer the definitive
solution to all your personalization problems. A/B testing tools, content management systems, eCommerce platforms, UGC
aggregators… almost every tool provides some level of personalization nowadays.

Unfortunately, without a clear personalization plan, it’s easy to get carried away by shiny object syndrome and try
every tool in hopes that it will solve all your problems. Do this enough times, and you’ll end up with a Frankenstein
monster of solutions that don’t integrate and annoy your customers with endless popups and broken UIs.

Before picking a tool or technology, decide **how** you want to personalize. You should do this, first and foremost, by
continuously referring back to your experiment goal and dimensions, but you should also lean on your brand’s strengths
and values. If you’re a fitness brand that’s all about community, you might want to leverage the power of user-generated
content, at least in *some* of your experiments. If you’re a luxury brand selling elevated essentials, perhaps you can
offer human-curated recommendations to your customers.

This last example brings us to the next point: don’t be afraid to do things that don’t scale, especially in the early
stages of your company. Many brands go from zero to a hundred by integrating deep learning models that are complex to
tweak and customize, and they don’t realize that they’re missing out on many lower-hanging opportunities. You’d be
surprised by how much you can accomplish with some basic customer segmentation and a human touch.

### Dimension

Once you know what you’re aiming for and how you want to get there, it should be easier to understand which dimensions
you want to personalize across. For example, you might want to personalize content based on geography, customer lifetime
value, previous purchases, demographics, or other factors.

It might help to group personalization dimensions into three big buckets:

- **Historical.** These are actions your customer has taken before. For instance, you might segment customers based on
  their AOV or purchase history.
- **Demographical.** These are traits that will always be true about your customer. For instance, you might segment
  customers based on geography, gender, or age.
- **Contextual.** These are actions your customer is taking now. For instance, you might segment customers based on
  their cart contents or session length.

When thinking about dimensions, don’t get limited by data availability: just because you don’t know your customers’ age,
it doesn’t mean you shouldn’t use it in a personalization experiment. Part of your personalization program should be
designing ways to collect or infer the information you need to carry out your experiments, which has the added benefit
of enriching your understanding of your customers over time.

You also don’t want to limit yourself to generic customer data: we find that the most successful personalization
programs use proprietary brand data to create win-win outcomes. If you sell pet food subscriptions, you might want to
curate your product catalog based on the customer’s pet type or age. If you sell sports equipment, you could send your
customers content specific to their favorite sports. Think about the different ways customers engage with your brand,
and try to design your experiments to optimize and improve those interactions.

### Touchpoint

With your goal, dimension and technique nailed down, it’s time to pick a personalization touchpoint. With this term,
we’re referring to the specific customer interaction you want to personalize. This will likely depend on your goal and
dimension: for example, if you want to increase AOV (goal) by looking at cart contents (dimension) to suggest upsells
(technique), you can only do this after the customer has added something to their cart or completed their purchase.

When deciding which touchpoint to personalize, we encourage brands to look beyond their product pages or on-site user
experience. Even with the average brand, there are so many ways your customers might interact with you and so many
opportunities to personalize those interactions. The most effective personalization programs are omnichannel and focus
on often-neglected interactions, such as transactional notifications, or create entirely new touchpoints, such as
sending customers a personalized card for their birthday.

Again, don’t get limited by your current touchpoints: if you want to send personalized email campaigns but are currently
not doing any email marketing, don’t be afraid to start now. If you think push notifications might resonate with your
customer base, why not give them a try?

The goal is to catch your customers when they need personalization even though they don’t know it, and leverage the
power of data to turn a mundane interaction into an intimate, personalized experience.

## Measuring Experiment Effectiveness

Designing and launching your experiment is only the first step. For an effective personalization program, you want to
validate the effectiveness of your experiment, double down on the ones that work, and kill the ones that don’t.

To do this, you must go into every experiment with a clear definition of success. This means answering the following
questions:

- What metric(s) do you expect the experiment to move, and in what ways?
- Are you already measuring the metrics you need? If not, how can you measure them? Do you need time to get a baseline
  for comparison?
- For how long will you let the experiment run after measuring the results?

### A/B Testing vs. Attribution Models

A special mention here goes to A/B testing: like with AI/ML, inexperienced founders often jump into A/B testing without
a clear understanding of the technique. While it’s tempting to run all personalization experiments as A/B tests,
sometimes this can be overkill or even detrimental: A/B tests require specific expertise, dedicated infrastructure, and
relatively high traffic volumes to reach statistical significance.

Typically, we only recommend A/B testing for large brands, running experiments where the results are highly uncertain,
and a clear control group can be established (e.g., customers that see the regular homepage vs. customers that see the
personalized homepage). In all other scenarios, a robust attribution model is a more reliable way to measure the
effectiveness of your experiment—even better when it’s paired with qualitative customer insights.

### Following Through on Experiment Validation

However you decide to validate your experiment, make sure it’s easy to roll back if needed and remember to look at the
data and take action accordingly! A recurring “experiment review” meeting with your team, where you review all
experiments that need to be validated and decide how to move forward, can do wonders to keep you accountable.

Finally, don’t be too rigid: if you expected a cross-selling experiment to drive $100K in revenue, but it only drove
$80K, it might still be worth keeping, depending on the effort you need to turn it into a permanent, first-class
capability. For some experiments, it might be helpful to establish different thresholds of success. Going back to our
example:

- Anything below $50K might be a clear failure.
- Anything between $50K and $100K might be up for discussion.
- Anything above $100K in incremental revenue might be a clear success.

Generally speaking, whenever you make decisions about your digital experience, you want to be data-informed, not
data-driven.

## Ready to Start Personalizing?

eCommerce personalization can create remarkable outcomes: when leveraged intentionally, personalization programs
increase revenue, customer loyalty, conversion rates, and lifetime value. It’s a tool that should be part of every
eCommerce store’s marketing strategy.

With the GDTD model, you have a streamlined process to design, launch and validate your personalization experiments.
It’s a high-impact tool that evolves with your brand as you scale, better understand your customers, and develop more
sophisticated ways to engage them.

If you have questions or need help with your brand’s personalization program,
[we’d love to hear from you.](https://nebulab.com/contact-us) Let’s start personalizing!

The GTDT Model for Intentional eCommerce Personalization

Stefano Sarioli

Your stakeholders are complaining that the app is slow. You work tirelessly to shave millisecond over millisecond off your server’s response time. Yet, at the end of the month, the stakeholders still complain about the suboptimal performance, to only later discover that they are using the application on a smartphone connection, inside a building with hundreds of other people using a smartphone, and while taking the elevator to move from one meeting room to the other.

This is just a story, but maybe you have experienced (or are experiencing) something similar in the real world and are looking for a more scientific approach to performance tuning. Perhaps you are here looking for an easy solution to improve your performance by just tweaking a single parameter in your configuration.

Unfortunately, most of the time, that’s simply not possible. All the overnight success stories you read about are the result of long experimentation, and they conveniently forget to mention all the failed attempts. Performance tuning is complex: it heavily depends on your application’s context, architecture, and [tech stack](https://nebulab.com/blog/ecommerce-tech-stack). To succeed, you must get your hands dirty and enter into a continuous measure/tune/verify cycle.

In this article, we will showcase a process for experimenting with the ideal performance setup for a [Solidus](https://solidus.io/)-based eCommerce application, as well as some interesting conclusions we’ve reached.

## Starting with Puma threads

When configuring a Puma-based Rails application, you can tune a lot of values to optimize its performance. Still, we can simplify our process by focusing on four major levers:

* The number of Puma workers

* The number of Puma threads

* The amount of CPU in your environment

* The amount of RAM in your environment

For now, let’s assume the amount of CPU and RAM and the number of Puma workers are constant, which is the most straightforward configuration you can have. We’ll talk about Puma workers and horizontal vs. vertical scaling in a separate article.

That means we should just focus on the number of threads, which allows us to keep things simple and straightforward. In general, when tuning performance, we recommend tweaking one parameter at a time: touching multiple levers simultaneously will lead to inconclusive results, and you might end up making things worse.

## Works on my machine!

For this experiment, we have run all tests locally. This allows us to have full control over environment in which the load tests are running. It also gives the Rails server plenty of CPU and RAM, preventing those from becoming a bottleneck in our testing.

Another advantage of running the tests locally is the almost complete lack of DB latency. In the real world, DB latency might impact the ideal number of threads, since each DB request will block its originating thread for a little longer. However, you can fine-tune this parameter in production. In this case, we’re more interested in getting a ballpark number that we can start with.

## The setup

In this experiment, we will optimize a Solidus-based eCommerce application, running PostgreSQL and Redis for data storage and caching/background jobs.

We’re using [k6.io](http://k6.io/) for load-testing because the open-source version is free, easy to set up and use, and it’s scriptable.

As a first step, let’s bootstrap a simple Solidus store. We won’t go through the setup because the process is already covered in the [official Solidus documentation](https://guides.solidus.io/getting-started/installing-solidus#in-a-new-app). You can find all the code in the [GitHub repo for this post](https://github.com/nebulab/repo-for-globpost-are-you-leaving-free-performance-on-the-table-).

We wrote a k6 script that simulates a complete visit to the website and performs a guest checkout. Note that this approach has some limitations: for example, we’re not testing the authentication system. However, it’s a good approximation of what a typical user journey would look like on my store.

As recommended by k6, we’re not testing asset delivery: in production, assets will most likely be served by a CDN, so testing them in a local environment would just muddy my data. Right now, we’re only interested in optimizing the application server.

We’ve decided to load-test the system with 50 concurrent users, each completing a single journey in my application. We’ve run the script against our Solidus application ten times, progressively increasing the number of Puma threads from 1 to 10.

## The results

Here are some graphs showing our results:

![Requests per second](https://cdn.sanity.io/images/kqgmf0g2/production/01b5ecdc96e1fa7859b614281aee8419c0ecfa39-600x371.svg?fit=clip)

![Request duration](https://cdn.sanity.io/images/kqgmf0g2/production/82090ce65420951346613c6f146c18b5e94ffa49-600x371.svg?fit=clip)

![Elapsed time](https://cdn.sanity.io/images/kqgmf0g2/production/88a998380731794a35b121d895e51122c54b74a0-600x371.svg?fit=clip)

Like most eCommerce applications, our Solidus store is a DB-heavy codebase, and it blocks each thread for a relatively long time. This is noticeable in our benchmark because the application is heavily underperforming with fewer than 4 threads.

It’s also interesting to note how the default Puma value of 5 threads is a good starting point for our application: anything beyond 5 threads is only giving us marginal gains. It also looks like performance is actually *decreasing* above 8 threads, which is probably due to the excessive thread context switching.

## Conclusions

k6 is a powerful tool with many applications: for instance, it could be used to enforce service-level objectives through automated performance tests.

When it comes to performance tuning, it’s always a good idea to put all decisions under the microscope. It was interesting to confirm that the default value of Puma workers is indeed a sane default. For a Solidus store, I would probably recommend bumping that number to 8 as a starting point to squeeze out each bit of performance.

However, the number of Puma threads is just the beginning. For instance, in a real-world performance tuning project, we should also:

* Calculate the amount of RAM I need to accommodate my ideal number of threads.

* Experiment with the number of Puma workers with 1 CPU.

* Experiment with the advantages of having a dedicated CPU versus a shared one.

* Compare different PaaS solutions to evaluate the pricing of our ideal setup.

* Experiment with horizontal and vertical scaling for our needs, creating to a well-defined scaling strategy.

As we’ve mentioned in the introduction to this article, there is no silver bullet for performance tuning: each project needs to account for the context, technical and business factors at play. However, by adopting a rigorous, scientific approach, you can tame the usual chaos of performance tuning, reaching your desired results with much less effort.

Over the next few weeks, we’ll continue to expand on this topic. *Stay tuned*!



Are You Leaving Free Rails Performance on the Table?

As funding for headless commerce platforms reached a staggering [$1.6 billion](https://www.forbes.com/sites/forbestechcouncil/2021/05/11/why-is-headless-commerce-getting-16-billion-in-funding/?sh=5322b28f513a) in 2021, these systems aren’t to be ignored. But how exactly do they compare to the so-called traditional “monolithic” eCommerce platforms, and when should you pick one over the other?

This guide will explore the headless commerce model, its benefits, how it differs from the all-in-one monolithic commerce system, and when it’s a good option for your brand.

## What is headless commerce and how does it work?

Headless commerce is an eCommerce architecture where the frontend layer, or presentation layer, is decoupled from the backend commerce functionality.

Before we continue, let’s review the key differences between the frontend and backend:

* **Frontend:** This is the user interface. Customers see and interact with the frontend, whether they’re accessing your eCommerce site, shopping app, digital kiosk or other shopping channels.

* **Backend:** The backend is the behind-the-scenes, technical side of an eCommerce store the customer doesn’t see. The backend includes the technologies and tools that run in the background to ensure customers can shop and [have smooth checkouts](/blog/one-page-checkout-with-spree).

In headless commerce, the frontend and backend systems communicate via Application Programming Interfaces (APIs). This approach allows you to update the frontend, the customer-facing part of your site, independently of the backend and even use different development frameworks. This separation of concerns offers several benefits which we’ll explore below.

Examples of headless eCommerce solutions are Commerce Layer, Swell, and Nacelle.


## 4 benefits of headless commerce

![Woman lying on a couch and using a laptop](https://cdn.sanity.io/images/kqgmf0g2/production/96998b94bc1edf8908d27f5af24b1a34e88c56a8-1200x797.jpg?fit=clip)

Platforms that adopt a headless architecture offer several benefits compared to monolithic, or so-called traditional eCommerce platforms. Here are the top four advantages.

### 1. Omnichannel capabilities

In an omnichannel eCommerce experience, customers can start shopping on one channel or device and transition seamlessly to a different channel while the brand retains key information: common examples are a customer completing their purchase online, but selecting in-store pickup, or a customer buying something in-store and having the order show up their online order history.

Headless commerce makes it easy to serve the [74% of shoppers](https://www.salesforce.com/content/dam/web/en_us/www/documents/research/salesforce-state-of-the-connected-customer-4th-ed.pdf) who use multiple channels to shop: because the same backend system can serve multiple storefronts, you can easily adapt customer touchpoints and shopping experiences to serve your eCommerce website, your mobile app, social media pages, and even physical locations. All will be completely integrated and feel seamless for your customers to interact with.

### 2. Technical flexibility

By decoupling backend and frontend, headless platforms allow your development team to pick the frontend stack, including third-party providers, that they’re most productive with. A headless architecture allows you to adopt your favorite content management system (CMS) to create rich editorial experiences, or elevate your storefront through progressive web apps (PWAs). This enables you to create engaging digital experiences that don’t face the limitations of traditional monolithic frontends.

When done right, headless architectures even allow you to refactor or completely replace entire parts of your backend with little to no impact on your frontend experience(s), which allows you to move and experiment much more quickly.

### 3. Superior personalization

Most monolithic eCommerce platforms severely limit your ability to personalize the customer experience by forcing you to work with proprietary templating language and preventing you from accessing customer data. With [80% of customers](https://www.epsilon.com/us/about-us/pressroom/new-epsilon-research-indicates-80-of-consumers-are-more-likely-to-make-a-purchase-when-brands-offer-personalized-experiences) indicating they prefer brands that offer tailored experiences, eCommerce personalization is now table stakes for modern brands.

Headless platforms allow you to not only leverage basic eCommerce data, such as a customer’s profile and order history, but even augment it with brand-specific data points of your own. This allows you to create shopping experiences that feel built for your customer, boosting customer engagement, conversion rates and AOV.

### 4. Blazing fast performance

Storefronts that leverage a headless eCommerce platform can typically be deployed as static sites and put behind a global content delivery network (CDN) such as Cloudflare. In the hands of a capable development team, this can lead to incredible performance that would be hard to achieve with a monolithic platform.

A 2019 study by Portent has found that improving your loading speeds can boost your conversion rates [anywhere from 1.5X to 2.5X](https://www.portent.com/blog/analytics/research-site-speed-hurting-everyones-revenue.htm)! If your eCommerce site loads slowly, you are leaving lots of money on the table—literally.

## 3 drawbacks of going headless

![Four programmers working at same office's table](https://cdn.sanity.io/images/kqgmf0g2/production/0afcfb4759febba635d79b5004c9a4cf4c8c2242-1200x800.jpg?fit=clip)

In some cases, a headless build can lead to an unnecessarily complex architecture that causes more problems than it solves. Evaluating which eCommerce platform to use for your business needs is a complex process that should take into account both short-term and long-term factors as well as your budget and internal capabilities.

Here are three drawbacks of headless eCommerce platforms.

### 1. Technical capabilities

Launching a headless storefront requires you to create an in-house engineering team or hire an eCommerce agency for the long term. Headless sites require more ongoing maintenance to keep the lights on: Dependencies need to be updated, infrastructure needs to be maintained, and the site needs to be continuously monitored for bugs and performance issues.

In our experience, this additional effort has its upsides: It forces brand operators to get into a continuous optimization mindset, where they’re constantly identifying and capitalizing on new technology and other opportunities that can improve the shopping experience. Monolithic platforms, on the other hand, encourage a less intentional set-and-forget approach, which is definitely easier to execute, but also worse positioned for long-term scalability.

### 2. Time-to-market (and cost!)

Headless builds are typically associated with slightly higher timelines than monolithic builds. Adopting a headless architecture requires a significant investment in an agency or an in-house development team that will build your storefront, and this process can take anywhere between 2 and 4 months, depending on the complexity of the build.

Conversely, launching a store on a monolithic platform can be a much faster process with more competitive pricing, if you’re willing to settle for a more standardized look and purchase experience. This can be an acceptable compromise for smaller mom-and-pop businesses, but certainly not for a direct-to-consumer (DTC) brand with large long-term aspirations.

### 3. Integration effort

Headless eCommerce platforms are very flexible because you have full freedom to implement your storefront however you want. Unfortunately, this also means that many of the integrations and plugins that you can use out of the box with a monolithic platform will require a bit more work to be integrated in a headless architecture.

If you want to add reviews to your headless eCommerce website, for instance, you’ll need to work with your designers and engineers to figure out what the experience should look like, where to place the reviews, when to ask for them etc., and then implement and test the new functionality.

Ultimately, this extra effort is a good thing: A headless system asks you to be more disciplined and intentional in how you modify your storefront. This will force you to approach all initiatives with a critical eye rather than jumping from trend to trend, but it can be frustrating if you’re used to installing a plugin and forgetting about it.


## The verdict: headless or monolithic?

![Woman holding a credit card and using a phone](https://cdn.sanity.io/images/kqgmf0g2/production/cb3e8d957a2c46acdf1eda32e9cecd9c5173bf13-1200x800.jpg?fit=clip)

The answer is: It depends on the use case. There’s no right or wrong answer.

While a headless commerce architecture has myriad benefits over monolithic architectures, it’s not ideal in all contexts. In certain scenarios, a headless commerce approach is the best way to build your vision and drive superior brand performance, while in others, you’re better off with a monolithic architecture that saves you time and money.

Headless commerce solutions might be right for:

1. **Brands that want to offer a consistent customer experience across channels:** Any brand that wants to deliver consistent customer experiences regardless of channels or devices should consider adopting this architecture.

2. **Brands that feel their current platform is too rigid and restrictive:** If you’ve ever felt your platform is limiting on the backend or frontend, it’s time to switch to a flexible headless setup.

3. **Brands that put personalization front and center:** If personalizing the shopping experience is critical for your brand’s strategy, then you should explore the capabilities that a headless build will unlock for your development team.

4. **Online stores that have slow-loading pages:** If sluggish pages have been a pain point for your eCommerce store, a headless architecture may be able to help by allowing you to leverage a more robust frontend ecosystem.

On the other hand, monolithic builds are a good solution for:

1. **Brands with limited technical know-how:** Headless commerce requires stronger technical capabilities which might not be everyone’s cup of tea. If you want to launch a commerce website without dealing with the underlying technicalities, headless is not for you.

2. **Brands with time-to-market or financial constraints:** If you’re on a tight timeline or a limited budget, a monolithic build will very often prove cheaper and quicker to launch than a headless counterpart.

3. **Brands that want ready-made functionality:** If you want to leverage ready-made features and integrations without thinking too deeply about how to integrate them in your shopping experience, a monolithic platform will give you more peace of mind.


## Need help picking the right platform?

![Entrepreneur using his phone and laptop](https://cdn.sanity.io/images/kqgmf0g2/production/97e367e0de824649a05070d381052e44e0ca20e4-1200x801.jpg?fit=clip)

Headless architectures allow teams to move faster, address changing consumer needs, improve the user experience, and capitalize on omnichannel opportunities, but they’re not ideal for all brands.

Nebulab can help your new digital brand pick the right eCommerce platform for a greenfield build, or help your established brand design the right tech stack for its next growth spurt. We offer full-service consulting throughout all stages of your eCommerce project.

[Reach out](/contact-us) to see how we can help you create a vision for your eCommerce experience.

Headless Commerce: The Good, The Bad, and The Ugly

Michela Frecchiami

**3D illustrations are a trend of web design in 2021:** reality and virtual reality are getting closer and closer. The combination of 3D shapes and effects in the 2D space is a great example of this trend in action. This is the right time to start learning something new, then!

At Nebulab, every employee has a bunch of hours to spend on personal projects, trying new tools, or studying to improve their skills and professionalism, so I decided to spend some hours of my [Investment Time](https://playbook.nebulab.com/personal-growth/investment-time/) developing a new skill that is useful for creating trendy illustrations and animations. I want to share my design process and what I’ve learned along the way.

## Spline 3D, nice to meet you!

I’ve always been fascinated by 3D design. Several years ago I tried Blender but I found it very complicated: the environment was very far from the applications that designers like me use, and since it wasn’t a must-to-have skill at that time, I gave up very soon.

This time I approached again to the three-dimensional design by using a brand new tool that makes 3D easier for a not-3D designer like me. This tool is [Spline](https://spline.design/): its interface is very similar to the majority of other design apps like **Figma**, for example. On the left side, there’s the layers panel while the tools used more frequently are on the top of the window. On the right side, you can find all the other options like Global Settings, Shape options, Camera Settings, etc.

## Initial phase: copy, try, repeat

When I start to develop a new skill, my approach is: learning the foundation, seeing how others do it, and playing with features and effects. Mostly, **having fun.**

Since Spline 3D was launched quite recently, you won’t find tons of tutorials on YouTube but there is a useful [documentation](https://docs.spline.design/) that gives you an overview of what (and how) you can do with Spline. Moreover, there’s a library where you can find illustrations and animations for playing with all the features.

![iPhone 12 3D mock-up, by Spline 3D](https://cdn.sanity.io/images/kqgmf0g2/production/025512f4b8e8134e5685c24024004f3f5c314035-1680x1027.jpg?fit=clip)
*This is the original mock-up of the Spline library.*

![iPhone 12 3D mock-up, customized with the Nebulab website and brand color](https://cdn.sanity.io/images/kqgmf0g2/production/35badac7709f3f9ec8613d94bbd9e1a112b21fcf-1680x1027.jpg?fit=clip)
*This is the customized version I did, starting with the mock-up above.*


### Camera operations

One of the first things I’ve learned is how to change views by using camera controls.

* **[Option / Alt] + drag** to rotate the screen (camera).

* **Press [Space] + drag** to move the camera around the canvas.

These controls allow you to simulate different views of your canvas and let you understand how three-dimensionality works.

### Materials

In Spline, there’re **7 types of Material** thus the Lighting layer: initially, I was wondering about having more options for creating awesome 3D color effects, but then I sooner understood that mixing just two or three material layers thus the Lighting one, lets you create a lot of combinations and you can easily find yourself spending two hours or more on it because you can’t decide which one you like most! That’s why I think is better to complete the illustration as the first thing, adding some animations eventually, and then focus on the colors and lights. The simplest three-dimensional effect is given by the **Lighting** layer (try to fill your object with a color and move the Lighting layer above).

![Spline 3D, lighting material](https://cdn.sanity.io/images/kqgmf0g2/production/26e42163d282f174007aa0b35014236c9a350ca1-1680x896.jpg?fit=clip)

Adding a gradient by the **Gradient** layer seems a bit flat to me; a more precise method for adding gradient (or depth) to your objects is the **Depth** layer.

![Spline 3D, gradient vs depth material](https://cdn.sanity.io/images/kqgmf0g2/production/4f53e9f9ce19327d998a4750da1fd08003197cd6-1680x997.jpg?fit=clip)
*Gradient material vs Depth.*

## Playing with Material for creating something cool

Another design trend in 2021 is **holographic and reflective elements.** You can create these two effects using the **Texture** material: import your holographic background (stock images or created using Photoshop), and edit the texture properties until the result satisfies you.

[![alt](https://cdn.loom.com/sessions/thumbnails/b987486fc5ec4739982150c9e97e13d8-with-play.gif)](https://cdn.loom.com/sessions/thumbnails/b987486fc5ec4739982150c9e97e13d8-00001.mp4)

*An example of how to use the Texture tool for creating the holographic effect.*

Use **Fresnel** for creating a reflective effect like this ring I did:

[![alt](https://cdn.loom.com/sessions/thumbnails/360368ebe5fd40e8b46bab7bc5a2f322-with-play.gif)](https://cdn.loom.com/sessions/thumbnails/360368ebe5fd40e8b46bab7bc5a2f322-00001.mp4)

For each Material layer, you can set the mode property and the layer opacity as you’d in Photoshop (normal, multiply, screen, overlay).

### Tricks that can save your time

* You can **copy/paste material** from an object to another. **(▲)**

* With the **Cloner** tool, it’s possible to clone an object and you can set the number of clones you want, the *Type* (how the clones are placed on the canvas - radial, linear, grid), and you can adjust the position, rotation, and scale them.

* With the **Visibility** mode, you can display the wireframe of a 3D object (and that might be a cool effect for your artwork!).

**(▲)** While I was writing this article, Spline has released the Alpha II version that introduces the web app and the **Material Assets** feature: now is possible to save Materials as assets on your project!

[![alt](https://cdn.loom.com/sessions/thumbnails/f99dd970a5574e93a80cb4cfa5362576-with-play.gif)](https://cdn.loom.com/sessions/thumbnails/f99dd970a5574e93a80cb4cfa5362576-00001.mp4)
*The new feature, Material Assets, works like Figma.*

## Starting from a blank canvas

After the discovery phase, I felt ready to start with something simple like an infinite rotation of the Nebulab logo as if it was an app icon. Spline 3D lets you **import SVG** files and after the Alpha II version was released, you can use the **Extrusion** tool to transform flat shapes into 3D objects.

![Spline 3D, extrusion](https://cdn.sanity.io/images/kqgmf0g2/production/d36903c4141a571eaaefb59987465a70673f8418-1680x1291.png?fit=clip)

![Nebulab 3D logo](https://cdn.sanity.io/images/kqgmf0g2/production/f23079e867385dba280f291061465ae64d65e9d1-3584x2240.jpg?fit=clip)

## Let’s move it!

Now that I have the Nebulab logo as a shiny bold blue cookie, I want to add a simple animation. I browsed on Dribbble looking for some inspiration and then I opted for a rotating animation with a final ease-out motion.

### Understanding States and Events

At the top of the right menu, you can find States and Events. Playing with animations with Spline is pretty simple (like everything else 😃) also because you can see the preview whenever you want, by clicking the Play button on the topbar tools.

![Spline 3D, the animation preview is always available](https://cdn.sanity.io/images/kqgmf0g2/production/330dcd7d5bf7cd1ff5585d336e2be6a5d1e32fb0-906x198.png?fit=clip)

To make my first animation, I need two states:

* *Base State*, is the initial position or style of an object.

* *State 1*, which changes the previous state values.

You can change the position and/or the aspect of your object. In this example, I rotated the entire logo on its axis.

[![alt](https://cdn.loom.com/sessions/thumbnails/ed27166d1b6d43ba8a1b1f7fc589c3f6-with-play.gif)](https://cdn.loom.com/sessions/thumbnails/ed27166d1b6d43ba8a1b1f7fc589c3f6-00001.mp4)

Now, these two states need a trigger that you can add by the **Event** panel. (**Be sure to have selected Base State** and then add a new event). **Select the type of trigger for the event**:

* *Mouse up, down, hover*, that you can use to create a button and you can even link to an external URL.

* *Key up, down*, that is the same as the mouse types but it requires adding the key you want that triggers.

* *Start*, the animation will start by itself.

* *Look at*, this reply the cool effect of moving objects when the user pointer passes over the active area.

* *Follow*, the object is linked to your user pointer as a magnet.

I wanted my animation fired automatically, so I selected the *Start type*. I’ve previously grouped all the shapes that compose the logo, so I had one object to select for the event that I was creating. While the state to select, is *State 1*.

![Spline 3D, states and events](https://cdn.sanity.io/images/kqgmf0g2/production/f094307692892d64a34817160faa61abc5ba65ae-1680x1050.png?fit=clip)

You can decide to repeat the animation infinitely, or just once. Then you can select the **Transition** type; if you know a bit of CSS these names should sound familiar to you: linear, ease-in, ease-out, ease-in-out, cubic (which is like the CSS *cubic-bezier*). I’ve played with these all but in the end, I opted for the Spring transition because I like damping effect on this kind of animation. About the Spring transition, you can play with its values since you won’t find the combination that satisfies you.

[![A video showing 3D animation configuration](https://cdn.loom.com/sessions/thumbnails/4adb76bc402d4b558d099dae004a6a2f-with-play.gif)](https://cdn.loom.com/sessions/thumbnails/4adb76bc402d4b558d099dae004a6a2f-00001.mp4)

And that’s it! You can apply different effects to every single object by adding *State 1* on it and then add the object to the main event by the ***“Add a new object to event”* button.**

[![A short video of the Nebulab logo 3D-animated](https://cdn.loom.com/sessions/thumbnails/ba16d29395e2458aa393761f06939fef-with-play.gif")](https://cdn.loom.com/sessions/thumbnails/ba16d29395e2458aa393761f06939fef-00001.mp4)
*This is the final animation*

## Don’t limit yourself

As I said before, what matters is having fun! Having the freedom to create whatever you like lets you experiment with a greater variety of features, and makes you more confident with the tool.

![3D artwork done with Spline 3D](https://cdn.sanity.io/images/kqgmf0g2/production/eefd0a35280995051e73c67e691e681383f8a038-1680x1260.jpg?fit=clip)
*This is my first 3D artwork in which I played with materials, colors, and shapes.*

Even if it seemed simple (not a “WOW” illustration), I didn’t want to limit my experiments to what is trendy at the moment. I just wanted to create something for myself, as an artist draws to express their feelings. Ok, well, maybe I’m exaggerating, but what drove me when I designed that artwork was just creativity, nothing more. I’m pretty sure that looking at it in a month, I’ll say to myself something like *“How could I be so proud of that illustrationMouse up, down, hover,”* and then I’ll be back to revisit it, trying to push my boundaries a bit more.

![The Nebulab Solar System in a 3D version](https://cdn.sanity.io/images/kqgmf0g2/production/6d1e6fabee08ed85aaa418b928d34822fae9785a-1680x1260.jpg?fit=clip)
*I’ve redesigned all the planets of the Nebulab website with Spline. In this illustration, I included them in the Solar System.*

![3D character done with Spline 3D](https://cdn.sanity.io/images/kqgmf0g2/production/7519216a0df3a358ebba7d983ede50e569d51b7b-1680x1150.jpg?fit=clip)
*My first 3D character, a tribute for the running Olympic games.*

## How 3D design supports your workflow

Even if you’re not a 3D artist (and if you’re reading this article I suppose you aren’t) knowing how to create 3D objects or scenes can be very useful. You can create tailored mock-ups for a mobile app, for example, or adding some catchy objects that interact with the user pointer on your website.

![The Blog page hero with some 3D objects](https://cdn.sanity.io/images/kqgmf0g2/production/abc58349dae18f451b8da3506d386b92cd952481-1680x1260.jpg?fit=clip)
*I’ve tried a white version of the Blog hero with some 3D objects that interact with the pointer.*
[See the sample in action.](https://www.loom.com/share/0192deb8d73b41fdb55ed361f0b1c3a8)

[![Final animation videoclip](https://cdn.loom.com/sessions/thumbnails/0384abffdff0454c9b94eda02ccc272e-with-play.gif)](https://cdn.loom.com/sessions/thumbnails/0384abffdff0454c9b94eda02ccc272e-00001.mp4)
*You can even use this to show off a new website or mobile app, with a trendy minimalistic animation like this one.*

Surely, there’re tons of mock-ups you can download and use but having the flexibility to create your own is extremely better, even in terms of marketing.

I’m happy to have spent some hours of my *Investment Time* learning Spline 3D! Truthfully, I’m still experimenting with it, but what I have done so far has given me an incentive to continue learning this modern, trendy tool and to go deeper into the 3D world.

Experimenting with Spline for 3D Design

Andrea Vassallo

In this article, we are going to show how you can create a [NextJs](https://nextjs.org/) project with [Storybook](https://storybook.js.org/) and [TailwindCSS](https://tailwindcss.com/).

Before starting, I want to explain who are the actors and why it should be better adding them to our frontend application.

### NextJS

[NextJS](https://nextjs.org/) doesn’t need a lot of presentation. If you are here, you probably already know what NextJs is and why we should use it to build robust applications.

NextJs is a great [React](https://it.reactjs.org/) frontend framework, and it’s mainly used to build SEO-friendly applications since you can use three different types of rendering/pre-rendering methods:

* SSG (Static-site generation) the HTML is generated at build time and will be reused on each request,

* SSR (Server-side rendering) the HTML is generated on each request using a node server, so the response is SEO-friendly.

* CSR (Client-side rendering) the normale React behavior, the response is simply Javascript run by the client.

### TailwindCSS

Tailwind is one of the most powerful libraries to build your application style using HTML classes. [TailwindCSS](https://tailwindcss.com/) implements a ton of dynamic classes which apply different style rules to your application, and it is fully customizable using the configuration file!

Initially it will take time to understand which are the correct classes you need to use to build your interface, but one of the coolest thing in Tailwind is its super documentation.

By the way, the 2.0 version has recently been released with a lot of new features.

### Storybook

[Storybook](https://storybook.js.org/) is a great library that is mainly used to render our application components outside the NextJs environment. This means that you can check your components UI without making real queries or fetching data. You can also figure out how the application components change based on the passed props.

Storybook is more powerful when used with well-organized components, for example, using [Atomic Design(https://bradfrost.com/blog/post/atomic-web-design/).

**Example**

You will probably create your custom `Button` component in your project that will be rendered differently based on the passed props or the fetched data. With Storybook, you can check all the UI changes applying custom props from the Storybook panel or stubbing all your requests.

### TL;DR

Here you can find the fully configurated [project](https://github.com/nebulab/nextjs-storybook-tailwindcss)

## Let’s start!

For this article, I won’t split the application into components to be more direct. It would be best to do it in a real application to avoid using the `index.js` page directly.

The best practice is writing components splitting them by context using a great file structure:

* React documentation about components: [link](https://it.reactjs.org/docs/components-and-props.html#extracting-components)
* Atomic Design pattern about components structure: [link](https://bradfrost.com/blog/post/atomic-web-design/)

### Create the NextJS app

First of all, run this command to create your NextJs project:

```
npx create-next-app --use-npm nextjs-storybook-tailwindcss
```

and navigate inside the project directory with `cd nextjs-storybook-tailwindcss`.

### Run your application

```
npm run dev
```

### Refactor the default code

The `create-next-app` command creates the project with some unnecessary configuration since we’ll use **TailwindCSS** to build our style.

Replace the content in `pages/index.js` with:

```
const Home = ({ article, showImage }) => {
  if (!article) return "The article wasn't found!"

  const {author, company, image_url, content} = article

  const Image = showImage ? (<img className="w-24 h-24 rounded-full mx-auto md:flex md:self-center"
    src={image_url} alt="A beautiful cat" width="200" height="200" />) : ''

  return (
    <div className="w-96 mx-auto">
      <div className="h-screen flex items-center">
        <figure className="lg:flex bg-gray-200 rounded-xl p-8 lg:p-4">
          {Image}
          <div className="pt-6 lg:p-4 text-center lg:text-left space-y-4">
            <p className="text-lg font-semibold">{content}</p>
            <figcaption className="font-medium">
              <div className="text-blue-600">{author}</div>
              <div className="text-gray-500">{company}</div>
            </figcaption>
          </div>
        </figure>
      </div>
    </div>
  )
}

Home.defaultProps = {
  showImage: true
}

export const getServerSideProps = async () => {
  const response = await fetch('http://localhost:3000/api/article')
  const data = await response.json()

  if (!data) return { props: {}}

  return {
    props: {
      article: data
    }
  }
}

export default Home
```

Add your first API:

```
touch pages/api/article.js
```
```
const Article = (_, res) => {
  res.statusCode = 200
  res.json({
    id: 1,
    author: 'Andrea Vassallo',
    company: 'Nebulab',
    image_url: 'http://placekitten.com/200/200',
    content: 'The cat is a domestic species of small carnivorous mammal.'
  })
}

export default Article
```

Remove the CSS file: `styles/Home.module.css`:

```
rm styles/Home.module.css
```

![Nextjs without style result](https://cdn.sanity.io/images/kqgmf0g2/production/cf125dfe5106852987ebfd778e133fe4566ef324-537x357.png?fit=clip)

![Man pointing thumb down](https://cdn.sanity.io/images/kqgmf0g2/production/f804d6533a925773c96bc9055cd7aae96d76c578-640x480.gif?fit=clip)

The result is an ugly and boring HTML page.

**Let’s go applying our awesome style.**

### Install TailwindCSS

*Why the documentation isn’t enough in this case?*

Because NextJS came up with a bunch of pre-configured services like Babel and [PostCSS](https://postcss.org/) and we have to TailwindCSS with them.*

Reading the [TailwindCSS documentation](https://tailwindcss.com/docs/installation), you’ll see that we need three different libraries to complete the installation:

* tailwindcss

* postcss

* autoprefixer

We don’t need to install postcss and the autoprefixer libraries since they are already installed in NextJs.

Run these commands to install TailwindCSS:

```
npm install --save-dev tailwindcss
```

Unfortunately, we can’t use `npx tailwind init` to create the configuration file since it’ll check if you have installed the `autoprefixer` library.

Let’s create the configuration file adding the `tailwind.config.js` file:

```
touch tailwind.config.js
```

with the default configuration:

```
module.exports = {
  purge: [
    // This is not present inside the default configuration
    // but it's good to build your production application
    // Read more about this here: https://tailwindcss.com/docs/installation#building-your-css
    './pages/**/*.js',
  ],
  darkMode: false,
  theme: {
    extend: {},
  },
  variants: {
    extend: {},
  },
  plugins: [],
}
```

### Configure PostCSS

To configure PostCSS we have to override the [NextJs default one](https://nextjs.org/docs/advanced-features/customizing-postcss-config):

Install the dependencies:

```
npm install --save-dev postcss-flexbugs-fixes postcss-preset-env
```

Create the `postcss.config.js` file:

```
touch postcss.config.js
```

and fill it with the default configuration as well:

```
module.exports = {
  plugins: {
    'postcss-flexbugs-fixes': {},
    'postcss-preset-env': {
      autoprefixer: {
        flexbox: 'no-2009',
      },
      stage: 3,
      features: {
        'custom-properties': false,
      },
    },
  },
}
```

Add the `tailwindcss` plugin to the top of the plugin list:

```
module.exports = {
  plugins: {
    tailwindcss: {},
    ...
  },
}
```

### Include Tailwind in your CSS

Add these three lines of code to the top of this file: `styles/globals.css`:

```
@tailwind base;
@tailwind components;
@tailwind utilities;
```

### We made it

Check your application homepage; the result should be similar to this one:

![Tailwind result](https://cdn.sanity.io/images/kqgmf0g2/production/8610b447fdcea90e798a8e33440326e498f8cbba-485x292.png?fit=clip)

![](https://cdn.sanity.io/images/kqgmf0g2/production/ef653b4c8ced24daaf9d3b865d207d11b7b905bb-700x394.gif?fit=clip)

## State of mind

From here on, it’ll be a breeze building beautiful component with Tailwind, but we need to answer a couple of questions about testing:

1. How is this component rendered when the article data isn’t found?

2. How can we check all the component variants at any moment without changing the code?

[Storybook](https://storybook.js.org/) will help us to accomplish both.

### Install Storybook

The default `npx sb init` command that is used inside the official documentation installs libraries that we don’t need and adds a bunch of files that we don’t use.

For that reason, I usually prefer to use the `from scratch` configuration.

```
npm install --save-dev @storybook/addon-actions @storybook/addon-essentials @storybook/addon-links @storybook/react
```

Add the Storybook commands inside the `package.json` scripts:


```
"scripts": {
  "dev": "next dev",
  "build": "next build",
  "start": "next start",
  "storybook": "start-storybook -p 6006",
  "storybook:build": "build-storybook"
}
```

Copy this piece of code inside the `.storybook/main.js` file

```
module.exports = {
  "stories": [
    // Paths to the story files
    "../pages/*.stories.mdx",
    "../pages/*.stories.js",
  ],
  "addons": [
    "@storybook/addon-links",
    "@storybook/addon-essentials"
  ]
}
```

Copy this piece of code inside the `.storybook/preview.js` file

```
// Import the global style enabling tailwind classes
import '../styles/globals.css'

export const parameters = {
  actions: { argTypesRegex: "^on[A-Z].*" },
}
```

### Add a story

For each page or component that we want to tell using a `story`, we need to create a file called `component-name.stories.js`.

In this case, we want to create the `index.stories.js` inside the `pages` dir:

```
import Article from './index';

const articleMock = {
  id: 3,
  author: 'Dave Canterbury',
  company: 'Bushcraft 101',
  image_url: 'https://via.placeholder.com/200/200',
  content: 'David Michael Canterbury born September 19, 1963 is a survival expert'
}

const Story = (props) => <Article {...props} />

// Here we export a variant of the default template passing props
export const ArticleStory = Story.bind({})
ArticleStory.args = {
  article: articleMock,
};

// Here we export a variant of the default template passing props
export const EmptyArticleStory = Story.bind({})
EmptyArticleStory.args = {
  article: null,
};

// Here we export the default component that
// will be used by Storybook to show it inside the sidebar
export default {
  title: 'Article',
  component: Article,
  argTypes: {
    showImage: { control: 'boolean' },
  },
};
```

### Run the Storybook server

```
npm run storybook
```

![Storybook result with style](https://cdn.sanity.io/images/kqgmf0g2/production/6b98a19deccff277aaa00d736aff13aa969a23ac-846x906.png?fit=clip)

At the time of writing there is a chance you might run into this [bug](https://github.com/storybookjs/storybook/issues/12668). You can fix it with this [solution](https://tailwindcss.com/docs/installation#post-css-7-compatibility-build).

## That's all

Your project is ready to be built using a robust framework and beautiful style with an easy way to check component behavior in isolation!

I hope that you found this article helpful and easy to read.

Do you need the [Typescript](https://www.typescriptlang.org/) version? Let us know with a comment.

### Follow up story - use TailwindCSS and Storybook with SCSS

NextJS side this isn’t a problem, you need just to install the `scss` library using `npm`.

```
npm install sass
```

Rename the `styles/globals.css` to `styles/globals.scss` and fix the `_app.js` and `.storybook/preview.js` style imports.

and replace the TailwindCSS style import from this:

```
@tailwind base;
@tailwind components;
@tailwind utilities;
```

to this:

```
@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";
```

**And Storybook?**

![Storybook SCSS error](https://cdn.sanity.io/images/kqgmf0g2/production/59838f648762570d289c5201a0bf7848015ae22e-1042x410.png?fit=clip)

It seems that Storybook doesn’t recognize the SCSS syntax, and for this reason we need to extend the library [Webpack](https://webpack.js.org/) configuration adding the `sass loader`.

The Webpack loader is used starting from the end, and this means that the `sass-loader` should be added at the end of the loader list.

Here you can find the `.storybook/main.js` file with the correct configuration:

```
module.exports = {
  stories: [
    // Paths to the story files
    "../pages/*.stories.mdx",
    "../pages/*.stories.js",
  ],
  addons: ["@storybook/addon-links", "@storybook/addon-essentials"],
  webpackFinal: async (config) => {
    config.module.rules.push({
      test: /\.scss$/,
      use: [
        "style-loader",
        "css-loader",
        "postcss-loader",
        // Add the sass loader to process scss files
        "sass-loader",
      ],
    })

    return config
  },
};
```

Using Tailwind and Storybook with Next.js

Let’s consider two engineering managers, John and Jane. They have the same seniority, they both work on similar projects in similar conditions and their teams are both terribly smart.

John insists on personally reviewing and approving every change, no matter how small. This doesn’t just concern the codebase: if an engineer wants to try a new productivity tool with the team, they have to go through John. If the team wants to move an internal meeting to a time that better suits everyone, she has to go through John. Every little detail is scrutinized and torn apart to make sure John’s team don’t waste their time on “useless” initiatives.

Jane, on the other hand, is more comfortable with experimenting. She doesn’t require the team to ask her approval for everything, but she is there to support them when they want to try something new, and helps them validate initiatives by setting clear goals and metrics that make sense for them. Then, she takes successful experiments and acts as the team’s champion in the larger organization, implementing these initiatives at a higher level.

Which team do you think will be happier, more productive and more innovative — John’s or Jane’s?

## Bottleneck Managers Create Permission-Centric Organizations

John is the typical example of a **Bottleneck Manager**. We all know at least a few Bottleneck Managers: they hoard information and authority and require teammates to go to them for every little thing. Want to try a new tool? Ask your manager. Want to move a meeting? Ask your manager. Want to change the name of a Slack channel? Ask your manager.

Put enough Bottleneck Managers together and you have an alienating workplace where there’s a process for everything and no way to change that process, until one day you realize you’ve just been doing the same thing for the past five years and you’ve become irrelevant two years ago anyway, but no one has told you because they were too busy writing code. You have, to put it briefly, a **Permission-Centric Organization**.

Permission-Centric Organizations are the natural result of one or more of the following factors:

* **Lack of trust**: his goes both ways. Managers may not trust their teams to make sensible decisions, or teams may not trust their managers to make good use of the results of their decision-making. If a teammate wants to try a new technology but knows their manager will scoff at it, they may not even propose it, to avoid the uncomfortable feeling of rejection.

* **Lack of time**: managers want to make sure everyone on their team is using their time in an optimal way. In doing so, they may overly centralize the decision-making process, to the point where the team is not free to do anything but write code. In some companies, managers will allocate all of the engineers’ time to “busywork”, leaving no room for anything else.

* **High cost of error**: if you have a proven process and battle-tested tools in place, it’s scary to try something new. Sure, the new tool/methodology may improve your work by an order of magnitude, but what if it doesn’t work and you end up alienating your stakeholders? The more successful you get, the more this becomes a problem.

* **Lack of competency**: when team members are not competent enough to experiment in a safe, productive way, management may create constraints around experimentation and take the reins of the engineering process. This may also come from the team members themselves: if they are not (or don’t feel like they are) competent enough, they may avoid trying new ideas for fear of making a mistake.

These are all valid problems. If you’re running a successful company, you want to make sure innovators are competent, you want to trust them and you want them to trust you, you want to dedicate time to the things that really matter, and you want to eliminate or minimize any room for mistake.

However, there’s a way to have your cake and eat it too. You can have all of the above, without hindering your team’s ability to come up with new ideas and validate them. The way to do it is to stop being a Bottleneck Manager and start being a **Multiplier Manager**.

## Multiplier Managers Create Feedback-Centric Organizations

Unlike John, Jane is a Multiplier Manager. A Multiplier Manager supports their team, but also allows them to make their own decisions. They’re proactive in identifying opportunities, but let the team capitalize on opportunities in their own way. They create processes and boundaries for safe innovation and encourage the team to pursue their ideas. They are — by far — the best kind of manager you’ll ever work with.

Multiplier Managers are the bread and butter of **Feedback-Centric Organizations**. These are workplaces where you can — and are strongly encouraged to — run an initiative by your manager to get feedback on it, without first having to ask permission. They are workplaces where ideas move freely from one desk to the other, in continuous evolution.

Just like a Bottleneck Manager and a Permission-Centric Organization are identified by a particular set of dysfunctions, Multiplier Managers and Feedback-Centric organizations exhibit and promote most (preferably all) of the following virtuous traits.

### Provide context about the business

Lack of transparency about the business context and business goals of their organization is one of the most common reasons why people don’t feel like they’re empowered to change their surroundings — withholding information may save your team some time, but it also prevents them from providing their perspective and putting their own twist to what you’re doing.

Multiplier Managers make sure to share all relevant information with their team. Not just what they’re doing, but why they’re doing it and whom they’re doing it for; any decisions that are made, and why they’re being made; who’s responsible for what; any processes the team follows and the rationale behind them, and anything else that might be of interest to other people.

Once the amount of information is too much to be pushed to all teammates all the time, they start publishing it instead and give their teammates an easy, clear way to access it, so that they don’t risk wasting the team’s time with anything they don’t find interesting. This may take different forms: a shared Google Doc, an entire shared folder, a company playbook, meeting recordings or a bunch of those — whatever works for the company and how people communicate.

## Define what “good” innovation looks like

This is where bureaucracy and innovation meet: the best approach to innovating is by doing it within clear constraints and with a streamlined process. Constraints give your team members a way to frame their ideas in a way that makes sense for everyone, and process lets them validate and execute their ideas in a controlled way.

Here’s the stuff that Multiplier Managers are concerned with:

* What kind of innovation is good innovation? Is it okay to propose rewriting a perfectly working project in a new, shinier technology? Is it completely forbidden, or should it just be a time-boxed endeavor?

* How do we determine if an idea is successful? Do we need to set clear business metrics in advance, or do we use more qualitative metrics? Does it depend on the context?

* What’s the process for collecting feedback? Even more important, what’s the process for understanding whom to collect feedback from in the first place? What kind of feedback should be asked for at what points during the process?
Your mileage may vary, but the mere act of creating a process will be beneficial to your team and make it known that not only is it okay to experiment, it’s encouraged and codified.

### Use automation to create safeguards

If your organization is run by humans, someone is bound to screw up at some point. While Bottleneck Managers run from mistakes and pretend everything can be perfect all the time, Multiplier Managers embrace chaos and try to minimize the ways in which things could go wrong, the cost of things going wrong, or both.

Multiplier Managers use automation to their advantage: they know that systems are dumb and infallible while people are smart and fallible. This leads them to give their team ample freedom to innovate and move quickly *without* having to ask permission. At the same time, they create automated safeguards around the critical aspects of the company’s infrastructure to protect them from human error.

The most obvious example of this is creating a CI pipeline and setting up the proper telemetry to ensure developers can iterate on the codebase as quickly as they can think, *without* running the risk of breaking the app with a bad deployment. Removing the human element from the approval process is key to boosting speed and creativity in the organization.

### Promote technical excellence and innovation

As you create more room for experimenting and taking ownership, you should also make sure you’re promoting technical excellence at all levels of the organization. Having the right skillset empowers individual contributors to spot and seize opportunities for improvement, and it empowers managers to recognize and support innovators.

Before you start firing people, though, make sure those “code monkeys” on your team are not monsters of your creation. Even the most proactive engineer in the world will stop coming up with new ideas when there’s no room for execution. Consider each team member’s history and their aspirations and, most importantly, *talk* to them about the kind of change you want to create to see how they react: are they excited, worried or indifferent?

If you’re trying to revive a stagnating team, a useful icebreaker may be to allocate time for intentional brainstorming and innovation. Try to put aside 20% or 10% of each iteration for the team to try their craziest ideas, and see how that pans out. This will give you time to get aligned on what innovation means to you, and how you want to go about it.

However, remember that this is still not ideal: “thinking time” is still you giving your team permission. Instead, you want to get to a point where they come to you spontaneously with their ideas, without waiting for you to tell them it’s okay to think. If you can get there and maintain that momentum, you’ll have a real Feedback-Centric Organization.

The Nebulab