It is inconsistent to be against growth or to advocate for social systems because you think they will reduce growth.
The most common formulation of this argument that I hear in the wild goes, essentially “capitalism bad because exponential growth is causing problems, do socialism instead” (ignoring the fact that the USSR grew faster than the US, for many years, and ignoring the benefits that exponential growth has brought). There are other problems with this argument, in my mind – if you plan on improving quality of life for people, how will you do that without growth of resource consumption? – but let’s address the general idea of advocating for a system on the grounds that it will grow slower.
Systems which grow slower than other systems will eventually not exist in any meaningful capacity. This is basic evolutionary theory – if there are two species, and one has kids twice as fast as the other, the fast-reproducing one will eventually take over entirely. The same is usually true of institutions – imagine two ice cream stores, one which cares about profit-maximization at the expense of all else, and the other which cares about X, for any X other than profit-maximization. Both ice cream stores want to become chains, and open a second location. The profit-maximizing ice cream store will maximize profit, and so earn enough money to open a second store faster, and then with its now doubled revenue (simplifying for this example), it will be able to open a third store before its competitor, and so on.
If you want to lower growth, you can’t do it by yourself. If any group in an evolutionary competition (and they’re all evolutionary competitions, in the long run) stops competing to maximize growth, they’ll become irrelevant. If you adopt a social system that intentionally limits growth, you will be out-grown by people who don’t.
That’s not to say that we are stuck maximizing growth. If growth was bad for everyone, you could potentially escape the growth-maximization dynamic with collective action. You could get countries to sign an agreement that would only take effect if enough countries signed, requiring any signatory to take military action against any economy that is growing too fast, however you want to define that. That would be impractical and difficult to enforce, but at least then you might have a chance of winning against the runaway growth you claim to want to fight.
Personally, I’m in favor of growth and progress. If you are anti-growth, all I ask is that your proposed solution won’t obviously fail in the long-term.