SpaceX progress towards a 100 launch year

[steveleach]

Here's a couple of charts of the actual monthly launch rates from Jan 2021 to Aug 2023, one with an exponential trend line fitted (by Google sheets) and one with a linear trend. All I've done with the grey dots is try to match those trend lines with a monthly growth rate.

Both trend lines go through the data, but the linear trend projects they'll get to 9 by the end of this year, and 11 by the end of next year. The exponential trend projects hitting 10 by the end of this year, and averaging 12 next year.

Read into this what you will.

I don't find this surprising.  Shouldn't we expect that the exponential trend will end up higher than the linear trend?

The real question is which trend line, the exponential or the linear, fits the data better.

And we want an objective measure of that.

There's probably some well known way to do this.  And I feel like I even knew it at some point.  But I can't remember it.

So I'll just make something up.  Take the absolute value of the distance, measured vertically, from the actual data points to the trend line.  Sum them all up.  Do it for both trends.

The trend line with the smaller sum is the better fit.

There are statistical confidence intervals and other tools to do just that, in a rigorous way. No need to reinvent the wheel with ad-hoc methods. A few months ago when things were still less clear I made such an argument with the so-called "R value", which clearly showed better fit to a linear trend within a yearly period. Some didn't want to understand it, muddling the waters by appealing to long-term trends or esotericities, even stating 120+ launches in 2023 was a good fit to the trend.

I won't take time to redo that analysis at the moment, given some pretty good probes have been derived here in the meantime, along with more rigorous mathematical principles. There is still an amount of uncertainty that gives wings to fantasies, but the evidence is now pretty unambiguous with 2/3rds of the year passed. To no detriment to Falcon's operations, which are no doubt jaw-dropping.

For what it's worth, I found a "show R²" button on the charts.

For the exponential trend line it is 0.663 and for the linear trend it is 0.673 which (I think) means that the linear trend is a slightly better fit (?)

I also tried going (yearly not monthly) back to 2012, and can get an exponential trend line with an annual growth of 34% out of it (significantly less than the 55% that best matches the current year) and an R² of 0.949; the linear trend gives negative launch rates in 2012 & 2013, so I'm not sure it is valuable.

[eeergo]

For what it's worth, I found a "show R²" button on the charts.

For the exponential trend line it is 0.663 and for the linear trend it is 0.673 which (I think) means that the linear trend is a slightly better fit (?)

I also tried going (yearly not monthly) back to 2012, and can get an exponential trend line with an annual growth of 34% out of it (significantly less than the 55% that best matches the current year) and an R² of 0.949; the linear trend gives negative launch rates in 2012 & 2013, so I'm not sure it is valuable.

That's it: the annual launch rate is compatible with a linear trend, within the noise, and has been so in recent years. An exponential trend is also compatible but slightly worse and with a very small exponent (i.e. approaching linear).

Moreover, the fine-tuning necessary to avoid over/under-shooting of an exponential function is much stronger, while multi-year-fitting exponentials that cover a few years back don't predict close to 100 launches this year, plus require piecemeal treatment of certain periods or tampering with the slope in approximately-yearly periods to fit the data well enough and not explode toward unfeasible values (like the linear trend does too, but subject to more error in case of inaccuracies in the fit).

The overall historical launch trend, while still noisy, is much more clearly exponential, as a simple visual inspection clearly tells you - although in reality it will taper off at some "saturation" value achieving something like a sigmoid, at least in the medium term.

[steveleach]

Musk said that they hope to make 10/month by the end of the year.  Whatever that means. 

They need four 10-launch months to make 100. 

Isn't it 9 per month for the remaining months that they would need to reach 100?

62 (now)
34 (4 x 9 less 2 already included in the count)
 4 (more Starship test flights)
--
100

Edit: Realized it was 62.

I'll be impressed if they manage 4 more Starship flights this year, to be honest.

But given they were 1 day out from 10 launches in August and managed 9 in May, I'm not going to bet against them launching 9 F9s in each of September, October and November. If they then get the 10 in December that Elon was hoping for, that gives 59 + 9 + 9 + 9 + 10 = 96 F9 launches for 2023. Add in 2 Starship launches (one historic, one in Sept) and you are at 98.

So if I was going to place a bet right now, it would be on 98.

[steveleach]

For what it's worth, I found a "show R²" button on the charts.

For the exponential trend line it is 0.663 and for the linear trend it is 0.673 which (I think) means that the linear trend is a slightly better fit (?)

I also tried going (yearly not monthly) back to 2012, and can get an exponential trend line with an annual growth of 34% out of it (significantly less than the 55% that best matches the current year) and an R² of 0.949; the linear trend gives negative launch rates in 2012 & 2013, so I'm not sure it is valuable.

That's it: the annual launch rate is compatible with a linear trend, within the noise, and has been so in recent years. An exponential trend is also compatible but slightly worse and with a very small exponent (i.e. approaching linear).

Moreover, the fine-tuning necessary to avoid over/under-shooting of an exponential function is much stronger, while multi-year-fitting exponentials that cover a few years back don't predict close to 100 launches this year, plus require piecemeal treatment of certain periods or tampering with the slope in approximately-yearly periods to fit the data well enough and not explode toward unfeasible values (like the linear trend does too, but subject to more error in case of inaccuracies in the fit).

The overall historical launch trend, while still noisy, is much more clearly exponential, as a simple visual inspection clearly tells you - although in reality it will taper off at some "saturation" value achieving something like a sigmoid, at least in the medium term.

The thing is, it looks like the growth is accelerating, not tapering off. 

From 2012 to 2022 it was trending at 35% a year.
From 2021 to 2023 it has been trending at 55% a year.

Is it common to see exponential growth turn into linear growth at a higher rate than it was while exponential?

[eeergo]

The thing is, it looks like the growth is accelerating, not tapering off. 

From 2012 to 2022 it was trending at 35% a year.
From 2021 to 2023 it has been trending at 55% a year.

Is it common to see exponential growth turn into linear growth at a higher rate than it was while exponential?

Linear fits are best in short (yearly) time periods because the rate doesn't change that much during that time, within the noise of singular launches. Attempting to fit to an exponential is actually overfit for these cases where any underlying exponential trend is over much longer timescales, and can lead to much larger errors when data is sparse. An exponential fit is clearly best for an all-history dataset, for now, until it becomes a saturated sigmoid.

But the point throughout the discussion in this thread was a counterargument against extrapolating linearly, which was demonstrated to be adequate for last year, and is turning out pretty fine for this one. The train of thought was that it was unreasonable to assume ~yearly increases in slope. But it's as unreasonable as trying to impose a (worse-fitting) exponential function -with more parameters- if one has to change its values every year too: there's little difference between a long-term exponential made up of year-long constant-cadence segments, and one made up of year-long, very slightly-increasing segments that are actually a worse fit to the data.

At the end of the day, this is a non-ideal system based on many stochastic and deterministic variables (i.e. weather, customers, technical problems, vs LV improvements that come online at a certain date, company directives decided at a certain moment in time...), which has shown significant departures from exponentials, or even linear functions, in past yearly periods. What you need for a faithful prediction of (near) future numbers is as simple a model as possible that is minimally sensitive to short term "noise" in the data. The more extreme parameters you add to that.

[Barley]

For the exponential trend line it is 0.663 and for the linear trend it is 0.673 which (I think) means that the linear trend is a slightly better fit (?)

What it means is that there is insufficient data to distinguish between the two hypotheses.

It's probably time to steal the joke about how you know an economist has a sense of humor: they use decimal points.

[alugobi]

Musk said that they hope to make 10/month by the end of the year.  Whatever that means. 

They need four 10-launch months to make 100. 

Isn't it 9 per month for the remaining months that they would need to reach 100?

62 (now)
34 (4 x 9 less 2 already included in the count)
 4 (more Starship test flights)
--
100

Edit: Realized it was 62.

Yeah, I wasn't clear there.  I was looking at my scorecard spreadsheet with 60 through August (including SS#1), with three already in September.  But I commented on what has to happen in those four months, namely, 10 each.  So seven more plus three 10-launch months. 

Whatever happens, this will be a banner year.

[steveleach]

But the point throughout the discussion in this thread was a counterargument against extrapolating linearly

Actually, I'd say it has mainly been countering the assertion that the growth rate is clearly not exponential. I don't know if anyone is still asserting that, given that we've now got R² values for both trends though, and they are pretty close.

I don't know about Robotbeat, but I've never really cared that much which model works out best, and you've seen that I've built both. I've just been resisting the insistence that an exponential model is wrong and that we should stop using it.

Right now I'm of the opinion that a linear fit for 2021-2023 is likely to under-predict the last 4 months of the year, whereas an exponential is likely to get it closer, but that's not statistics, that's crystal ball gazing (by me). The linear fit (Jan 21 to Aug 23) projects 91 (F9/FH) total for 2023, the exponential projects 96.

I'm not sure why some are saying that a linear projection to 85 for the year is evidence for that model, when it it would require negative growth for the remaining 4 months.

[FutureSpaceTourist]

Elon tweeted about the target increased monthly launch rate this year (10) and next (12), which i assume is what this relates to:

https://twitter.com/whoisheartbreak/status/1698802673473245256

I’ve been building rockets for 8.5 years now (yes the half a year makes a huge difference) in Falcon production, owning and building different parts and processes across 1st and 2nd stage.

We have always made really amazing strides, but the past 2 years we’ve made significant step changes in how we build rockets. From my own direct vision, solutions and contributions, to my team’s creative and impressive execution, to the whole Falcon production and launch program’s ability to raise the bar higher and higher.

I think the results speak for themselves, but i think the wildest part is we are not even close to being “done.” It feels more like we are just getting started..We have even more challenges to overcome and goals to achieve that once again are seemingly “impossible.” This is what keeps me excited for the future. 👩🏽‍🚀🚀

[steveleach]

The train of thought was that it was unreasonable to assume ~yearly increases in slope. But it's as unreasonable as trying to impose a (worse-fitting) exponential function -with more parameters- if one has to change its values every year too: there's little difference between a long-term exponential made up of year-long constant-cadence segments, and one made up of year-long, very slightly-increasing segments that are actually a worse fit to the data.

On this one, I think the assertion was that calendar year boundaries are very arbitrary places to put the breaks. If nothing else they make your projections almost useless at the start of the year.

And I don't think anyone has ever suggested year-long exponential segments. Robotbeat has used a 2022-23 trend (only), and I've used a 2021-23 trend and a (separate) 2012-2022 trend, and acknowledged that there is a mismatch.

Maybe Jessie's "but the past 2 years we’ve made significant step changes in how we build rockets" (post above this one) is what's behind that mismatch though. If so, that's not a change in the model, its a change in the system being modelled.

[xyv]

Exactly right.  Everybody is trying to fit a math model to a trend that has underlying factors that don't follow smooth functions and are compounded by noise.  "Jumps" in capacity occur sporadically; weather, customer payload rediness and other random events add significant noise.  No math model can be correct - it's a backward looking curve fit - extrapolate at your own risk.

I am score keeping and the natural comparison ("are we behind or ahead?") is the straight line to the end of year goal/bold claim.  For prediction, the current linear curve fit, forward predicted to the end of the year is a really nicely filtered and damped function.  Would be fun someone (not me - but it would be fun for me to read) to plot the same end-of-year preediction versus launches to date using an exponential fit.  I am going to guess that it will be a bit noisier than the linear forward projection.

[meekGee]

“Just increasing” is not a model.

No one is upset.

Nobody said it was.  It's a true statement though.

"Exponential" a wrong model. If the rate of increase is decreasing, it's not an exponent.

And the rate of increase is decreasing by a LOT.

I suggested that "linearly increasing" is a better fit.

[Robotbeat]

...
"Exponential" a wrong model. If the rate of increase is decreasing, it's not an exponent.

And the rate of increase is decreasing by a LOT.
...

Based on what? I've gotten about the same estimated number of launches since I started doing an exponential fit like 5-6 months ago. 100-105.

I don't expect the current exponential increase to hold past this year. They were targeting an increased rate, and because you can't flip a switch and just start launching at a higher launch rate starting January 1st, you have to model the increase in launch rate somehow, and a gradual compounding improvement is basically the simplest of models for that. It's a simple assumption that won't hold forever, but it sure as heck beats piecewise linear.

[meekGee]

...
"Exponential" a wrong model. If the rate of increase is decreasing, it's not an exponent.

And the rate of increase is decreasing by a LOT.
...

Based on what? I've gotten about the same estimated number of launches since I started doing an exponential fit like 5-6 months ago. 100-105.

I don't expect the current exponential increase to hold past this year. They were targeting an increased rate, and because you can't flip a switch and just start launching at a higher launch rate starting January 1st, you have to model the increase in launch rate somehow, and a gradual compounding improvement is basically the simplest of models for that. It's a simple assumption that won't hold forever, but it sure as heck beats piecewise linear.

You need to decide if you're looking long or short term.

For 7 months, the monthly total was exactly fixed.  Zero growth. And since this was a temporary condition, any analysis of a period comprising these 7 month plus the more recent increase is just nonsensical.

Looking back over a longer time period, 2021->2022  showed a huge increase (100%), 2022->2023 is looking like a little more than half that (50-60%) and 2023->2024 is forecast by Musk to be only a 20% increase.

If your curve fitting methodology tells you there's an exponent there, question the methodology...

i tried explaining why it's misleading but this gets lost in the back and forth so I gave that up.

[meekGee]

...
"Exponential" a wrong model. If the rate of increase is decreasing, it's not an exponent.

And the rate of increase is decreasing by a LOT.
...

Based on what? I've gotten about the same estimated number of launches since I started doing an exponential fit like 5-6 months ago. 100-105.

I don't expect the current exponential increase to hold past this year. They were targeting an increased rate, and because you can't flip a switch and just start launching at a higher launch rate starting January 1st, you have to model the increase in launch rate somehow, and a gradual compounding improvement is basically the simplest of models for that. It's a simple assumption that won't hold forever, but it sure as heck beats piecewise linear.

Based on what? Based on number of flights...

31
60
90-100  estimated
120  projected by Spacex

That's +100%, +50-60%, +20-33%

This doesn't change when you look at monthly rates, it's just that the yearly rates are less noisy.  Just have to remember that there's growth within a year, not just at each year's end.

[eeergo]

...
"Exponential" a wrong model. If the rate of increase is decreasing, it's not an exponent.

And the rate of increase is decreasing by a LOT.
...

Based on what? I've gotten about the same estimated number of launches since I started doing an exponential fit like 5-6 months ago. 100-105.

Again: no, you did not. As late as June 27th (i.e. just over two months ago) you were predicting upwards of 108, with as many as 110 in May.

And again: the exponential trend that best encompasses all available historical data currently predicts around 85-90 launches this year, as derived by AnalogMan a few months ago. Any other exponential predicting more accounts for significant corrections with correspondingly larger errors.

[Robotbeat]

Then let’s find out.

[steveleach]

...
"Exponential" a wrong model. If the rate of increase is decreasing, it's not an exponent.

And the rate of increase is decreasing by a LOT.
...

Based on what? I've gotten about the same estimated number of launches since I started doing an exponential fit like 5-6 months ago. 100-105.

I don't expect the current exponential increase to hold past this year. They were targeting an increased rate, and because you can't flip a switch and just start launching at a higher launch rate starting January 1st, you have to model the increase in launch rate somehow, and a gradual compounding improvement is basically the simplest of models for that. It's a simple assumption that won't hold forever, but it sure as heck beats piecewise linear.

Based on what? Based on number of flights...

31
60
90-100  estimated
120  projected by Spacex

That's +100%, +50-60%, +20-33%

This doesn't change when you look at monthly rates, it's just that the yearly rates are less noisy.  Just have to remember that there's growth within a year, not just at each year's end.

This is what the yearly rates look like...

[steveleach]

For 7 months, the monthly total was exactly fixed.  Zero growth.

What do you mean by that exactly?

Here's the data from Jan 21 with the first 7 months of 2023 highlighted.

[steveleach]

Would be fun someone (not me - but it would be fun for me to read) to plot the same end-of-year preediction versus launches to date using an exponential fit.  I am going to guess that it will be a bit noisier than the linear forward projection.

I thought this is what I'd done (for both linear and exponential) above. Happy to try to expand on it if it helps.