Basically if you have ever held a backstrip-less late 19th century book in your hands you know that these products of the industrial revolution had newsprint for hollowbacks and strips of ticking fabric for headbands.
Of course you can buy ready made headband yardage, but that seems like a waste of money. I just keep an eye peeled for finely striped cotton fabric and stow it in my bottom drawer. A half yard will last for decades.
Generally I cut a 2″ x 3″ swatch, then soak it in laundry starch, wheat starch, methyl cellulose, something that will make it nice and firm when it dries. Then fold it half over a piece of cotton string. squeeze out the excess moisture and let it dry, on a piece of wax paper or plastic. When dry, peel it up and use it to replace missing old ones. And btw don’t forget to dye it with tea or coffee to give it that century old dirty color.