Stiffness testing: Colnago Y1Rs vs Cervélo S5, Specialized Tarmac vs Allez Sprint, steel vs carbon, budget vs expensive and much more

Frame stiffness, both in terms of lateral stiffness and vertical compliance, is an incredibly important consideration when it comes to engineering a bike frame. It is the special sauce that defines how well a bike rides.

So much so that when brands are designing their latest flagship race bikes, it's given the same importance as aerodynamics and weight. Some will chase new stiffness targets, but others will have these figures set in stone as immovable targets from the beginning, as they aim to retain the DNA of the platform while improving its performance elsewhere.

Despite this, it's something of a mysterious metric. Short of looking at a frame and assuming its deep tubes will make for a stiffer ride, there's no way of knowing exactly what you'll get.

But given that carbon fibre is anisotropic in nature, meaning its mechanical properties vary depending on the direction of the fibres, two visually identical frames could have completely opposing characteristics based on how the carbon has been laid into the mould.

So for most of us, we're reliant on either actually riding a bike in order to gauge its stiffness and compliance, or reading a review from someone who has. Even then, beyond subjective feeling and opinion, there's no straightforward metric to allow you to compare one bike to the next.

That, hopefully, is where we come in. Here, with the advent of Cyclingnews Labs empirical testing, were dedicated to giving our readers as much objective information as possible, decoding brands' marketing claims, and offering fair, unbiased tests to help you decide on the next best road bike for you.

We've spent more hours than I care to think of in the wind tunnel testing the aerodynamic efficiency of bikes, wheels, clothing and more. We've tested tyre rolling resistance too, and now it's time to turn to frame stiffness.

This will centre on two measurements, the torsional stiffness at the bottom bracket and the vertical stiffness at the saddle.

It's not a simple case of more bottom bracket stiffness is better, or more vertical compliance is better, but the information below will hopefully add clarity to an otherwise vague understanding of bike stiffness and provide another piece of the puzzle to help you buy the right bike.

What is stiffness and compliance?

It's common for bikes to launch with claims of being "laterally stiff" and "vertically compliant," but what does that actually mean?

Starting with 'laterally stiff', this refers to the frame’s resistance to side-to-side deflection at the bottom bracket shell when under the 'load' of the rider's pedalling force.

Given the pedals are positioned on either side of the bike – typically around 75mm from the centreline of the frame – when a rider applies force to the right pedal, the frame is not only pushed downwards, but also forced to flex to the left, and then vice versa when applying force to the left pedal.

The conventional theory is that more flex means less of that force is being transferred to the drivetrain - and in turn to the rear wheel - whereas a stiffer frame should, in theory, be more responsive under power. However, some argue that the flex is strain energy stored as the frame acts like a spring, and the energy is returned to the drivetrain as the frame springs back to the centreline later in the pedal stroke. The timing of this return is where things get more complicated, though.

It's generally accepted that more stiffness is better, but only to a point, and the tipping point is rider-dependent.

The term 'vertically compliant', meanwhile, refers to a bike's ability to absorb small bumps and vibrations from the road, in turn mitigating the energy dissipated as heat when a rider’s body is forced to absorb the bumps, which accelerates rider fatigue.

This can be mitigated somewhat by fitting wider tyres, but many brands will design compliance into the frame in a way that decouples it from the lateral stiffness. This may be achieved simply by taking advantage of carbon fibre's anisotropic nature, but dropped seatstays are a common road-smoothing inclusion, leaf spring tube shapes too, while more active tech such as Trek's IsoSpeed is slightly rarer.

How we're testing

Our new test protocol is designed to quantify brands' claims in a way that can be compared fairly and consistently.

Primarily, we want to allow readers to compare across brands, such as pitting the Colnago Y1Rs against the Cervélo S5; but also across models within a brand, such as the Specialized Tarmac SL8 against the Allez Sprint; and eventually even across genres, such as from road bikes to gravel bikes, although we've focused on road bikes for now.

Both tests are undertaken at the same facility at the Silverstone Sports Engineering Hub, and are designed to match the setup for the ISO fatigue tests.

For both tests, a 'settling run' is performed to ensure everything is set up properly, such as ensuring the chain is secure on the chainring, the frame is settled into the jig correctly, any slippage in the seatpost can be identified, and box-fresh frames' fibres and resins are stressed.

Then, after a return to zero – well, actually past zero to -50Nm – each test proper is performed three times. If anomalous results are found, such as one of the three being wildly different to the others, the bike is removed, remounted, and retested three more times.

The figures we report below are the average of the three runs.

The details of each test are as follows.

Bottom bracket

To measure lateral stiffness, we remove the wheels and mount the bike via its thru axles to a fixed rig. A 'control' Shimano Ultegra crank is fitted to all bikes, the driveside crank arm positioned at 45°, and a load cell connected to a custom pedal spindle fitted to the driveside crank. A highly force resistant track chain is wrapped around the big chainring and connected to the rig to lock the crank in position.

A force is applied to the load cell, pulling the driveside pedal downwards. Throughout this, the lateral displacement of the bottom bracket is measured, in millimetres, as the force is slowly increased to 1100nm.

The resulting measurement is then given in newtons per millimetre (N/mm), effectively quantifying the number of newtons required to move the frame by each millimetre. The higher this number, the greater the frame's stiffness.

We then return the load cell to zero, repeat the process twice, and take an average of the three tests.

Seatpost

To measure vertical compliance, the same rig is used to fix the bike into place at the thru axles. The seatpost is set to height, the saddle removed, and a special mount fitted to the saddle clamp in place of the saddle.

Everything is torqued per the manufacturer's spec to avoid slippage, and the load cell is attached to the aforementioned saddle clamp mount. A force of 1000nm is then applied, pulling the seatpost vertically downwards.

Given that seat tubes are often around 74°, rather than vertical, the seatpost and frame will flex towards the rear. As the load builds, the displacement is measured, and as above, the result is given in newtons per millimetre (N/mm), quantifying the amount of force required to move the saddle by each millimetre.

Standardisations and controls

We ensure all bikes are built with fork and stem/cockpit fitted, and the necessary preload applied to the headset to ensure no undue movement in the system that could affect the results. Wheels are removed for fitting into the rig.

For the lateral stiffness test, each bike is tested with the same Shimano Ultegra FC-R8100 chainset, with 170mm crank arms.

For the vertical compliance test, the seatpost of each bike was set to a matching height, measuring 730mm between the bottom bracket centre and saddle rails centre.

To standardise the saddle position, we simply pushed the mount as far forward on the rails as possible (like a saddle being slammed forward). This left no ambiguity around the exact position across bikes and led to a 'worst case' score for each bike, since moving the mount further back would increase the horizontal distance from the bottom bracket and, in turn, the amount of flex you get. It is not valid to compare these figures to tests performed elsewhere, or even by other clients of Silverstone Sports Engineering Hub, since the seatpost height or lateral offset are unlikely to match.

Caveats

As ever, our data is simply a result of our day of testing using a set protocol, and not the final word. While we are confident in our data, different protocols will probably return different results.

Two identical bikes with different length seatposts would likely have different vertical compliance scores, since there'd be a different amount of material there to work against the force. Since nobody is cutting down their seatpost to improve comfort, we just run each bike as they are sold, rather than standardising seatpost length, as this is more representative of the bike you would end up with as a customer.

And similar to the above, a bike with a lower standover height would have more seatpost on show, and thus would likely be more compliant, all else being equal. Again, this translates to real-world use, so we've not tried to control it.

Given we're pulling the load vertically downwards, a steeper seat tube angle will technically be at a disadvantage compared to a slacker seat tube.

Matching the bottom bracket offset may have been a fairer test, but we didn't have a tool on hand that would measure this accurately enough, so we opted to control the saddle in relation to the clamp and quantify each bike in its least compliant setup. With more time, we'd do both, and perhaps also test each bike with the bracket slammed rearwards too, to get a min and max value, but we opted in this case to spend less time on each bike, testing more bikes in total, to get a bigger spread of the values we can expect to see in future tests.

The bikes

Unlike our wind tunnel tests in which we've generally tried to group bikes of a similar category together into each test – such as the superbike test, which focused on the bikes from the WorldTour, or the aero bike test, which looked at the fastest bikes on the market – here we've gone for a broader mix. We have lightweight and aero, budget and premium, steel, carbon and alloy too.

The reason for this is that you can't really quantify X stiffness = Y benefit in the same way that you can equate a CdA saving to a difference in watts required.

Comparing 10 super bikes in this context risked falling foul of selection bias, where the differences between each bike are amplified by the lack of the bigger picture; a baseline and a topline.

As it transpires, this decision actually gave us a bigger understanding of the subject matter than we expected. We presumed race bikes would be stiff, cheap bikes would be flexy, and steel bikes would set the baseline for comfort. We were wrong.

The bikes we tested are listed below, each in a size 56cm or nearest equivalent.

• Ridley Noah Fast 3.0
• Specialized Allez Sprint
• Giant TCR Advanced 0
• Cervelo S5
• Canyon Aeroad CFR
• Giant Contend SL Disc
• Trek Madone SLR Gen 8
• Van Rysel RCR F
• Specialized S-Works Tarmac SL8
• Specialized Aethos
• X-LAB AD9
• Colnago Y1Rs
• Fairlight Strael 4.0 (BB test only)
• Seka Spear RDC (BB test only)

Confidence interval

We ran the Specialized Aethos through both tests at the start of the day, in the middle of the day, and again at the end of the day to quantify a margin of error as we worked through the bikes and tested.

In the bottom bracket stiffness test, it resulted in an error margin of 0.48 N/mm in a dataset that ranged from 68.65 n/mm to 101.35, or 0.58% of the mean result.

In the seatpost stiffness test, we landed on an error margin of 34.67 N/mm, in a dataset that ranged from 632.89 N/mm to 1661.51 N/mm, or 3.9% of the mean result.

Results

Results: Bottom bracket

Starting with the bottom bracket test, the graph above shows each bike, ordered from least to most stiff. The minimum and maximum value generated by our confidence interval is overlaid in green, and the N/mm result is written within each respective bar.

Takeaways include the comparative extreme stiffness offered by the Ridley Noah Fast 3, and the high levels of flex found in the Seka Spear and Colnago Y1Rs.

The Fairlight Strael is the only steel bike we tested.

The Specialized Allez Sprint and Giant Contend SL1 were the only aluminium frames, but they have very different primary focuses. The Contend is aimed more at the entry-level market, while the Allez Sprint is aimed as a more budget-friendly race bike.

At first, we were surprised to see both the Specialized S-Works Tarmac (with higher-spec Fact 12r carbon) and Specialized Aethos (with Fact 10r carbon) so close in performance, but speaking to the brand afterwards, the brand outlined that the two were designed with very similar ride quality characteristics, which naturally includes a predetermined BB stiffness target.

What's more, a lower modulus carbon fibre doesn't mean less stiffness can be achieved, just that it takes more carbon to achieve it. This is why lower modulus frames often weigh more, despite coming out of the same mould.

Results: Seatpost

Next up, looking at the seatpost stiffness, the graph above again shows each bike ordered from least to most stiff.

Unfortunately, we overestimated the number of bikes we'd get through in the day, so the Seka and Fairlight didn't get tested.

The N/mm is once again listed within each respective bar, and the error margin is shown in green.

The exact N/mm figures here are around 10x higher than at the bottom bracket, simply because the amount of movement is smaller, and thus, the amount of force needed to achieve each millimetre of movement is greater.

We experienced a higher variance between our Aethos repeats in this test, for reasons unknown, and thus our error margin is greater.

That aside, there is still a clear difference between bikes, with the Van Rysel RCR-F at the stiffer end, with the Cervélo not far behind it. Meanwhile, the S-Works Tarmac SL8 and Aethos prop up the more comfortable end of the scale.

The Ridley's performance here is an interesting one. It is the stiffest in our bottom bracket test, but sits at the more supple end of the spectrum in terms of road comfort, perhaps landing a best of both worlds result.

Results: Combined

I maintain that the ideal stiffness at the bottom bracket, and the ideal compliance at the saddle, are both personal preferences based on rider weight, power output, riding style, and typical ride surface. I also believe that the preferences are independent of each other.

Regardless, I believe it's useful to see the data combined. If you know the charcteristics you want, bouncing between the two bar charts above to see how each bike stacks up in each test makes for a laborious process. This scatter plot should make that easier.

It highlights how opposing the ride characteristics of the Van Rysel RCR-F and the Ridley Noah Fast 3.0 are, which is notable given they share a similar visual silhouette and were so close in our wind tunnel tests.

Conclusions

While it's difficult, impossible even, to quantify exactly how much faster, slower or more comfortable you'll be as a result of the data above, I think there's still a lot that can be gleaned.

The Ridley is the bike that stands out with good results in both, but only if you believe more is better when it comes to bottom bracket stiffness. It's not necessarily as simple as that, of course, hence the rider-dependent tipping point mentioned earlier.

The Giant Contend's bottom bracket stiffness was perhaps my biggest surprise result. Although my initial assumptions that I'd find a correlation between price and stiffness were wrong, I still think more testing is needed (and I have every intention of doing so). I don't know for sure, and Giant's engineers are yet to reply to my questions, but I suspect that the bike was designed more to hit a price point, and that the stiffness is more a byproduct of the materials used, not a target iterated towards.

TCR was a second-tier frame. Giant actively promotes the TCR range's stiffness to weight ratio, and this seems to affirm that. We tried to get a Propel too, but were hampered by customs blocking it for just long enough to miss our testing day.

I've ridden many of these bikes, but to take the Ridley as a clear example, I could have told you that it's incredibly stiff under power, yet still comfortable to ride. The data finally adds some objectivity to that perception, showing how it compares on a measurable scale.

In time, with a bigger database of bikes tested, we hope to build a bigger picture of where all bikes sit on this scale and how different models compare to their peers.

And when combined with our wind tunnel bike tests, I hope our empirical, objective and impartial testing can help our readers make the right choice when buying their next bike.